Panoramic image splicing method for electronic guide rubber-tyred vehicle and related equipment

Abstract

The invention is suitable for the technical field of vehicle safety assistance, and provides an electronic guide rubber-tyred vehicle panoramic image splicing method and system, and the method comprises the steps: obtaining a to-be-spliced image through a preset camera disposed on each carriage of an electronic guide rubber-tyred vehicle, acquiring deviation angle data among different carriages through angle sensors arranged on hinge plates of the electronic guide rubber-tyred vehicle; acquiring image features of the to-be-spliced image through an adversarial generative network; and splicing the image features according to the deviation angle data to obtain a panoramic image of the electronic guide rubber-tyred vehicle. The to-be-spliced image is processed through the generative adversarial network, the definition of the panoramic image of the electronic guide rubber-tyred vehicle is improved, the image features are spliced according to the deviation angle data, distortion of the panoramic image is avoided, and the stability of the panoramic image is ensured.

Description

Technical Field

[0001] The present application belongs to the field of vehicle safety assistance technology, and in particular, relates to a panoramic image stitching method for an electronically guided rubber-wheeled vehicle and related equipment. Background Art

[0002] The electronic guided rubber-tyred vehicle is a fully electric-driven rail transit product and a new urban rail transit system independently pioneered by China. It is a new type of green transportation tool with "zero pollution" and "zero emissions". The electronic guided rubber-tyred vehicle is a multi-unit public transportation tool with medium and low capacity. It can effectively alleviate the pressure of urban public transportation, further optimize the urban spatial structure, improve the quality of the city, facilitate citizens' travel, and improve the city's happiness index. Compared with other public transportation tools, the electronic guided rubber-tyred vehicle is more intelligent and equipped with an automatic tracking control system and an environmental perception system. The electronic guided rubber-tyred vehicle is a multi-unit vehicle with a long body and a large blind spot. Therefore, the environmental perception system is particularly important. The environmental perception system includes various radars and cameras to obtain information about the vehicle. The driver and passengers can understand the situation around the vehicle through image information. Therefore, in the environmental perception system, the panoramic image of the vehicle is the top priority.

[0003] The existing technology is to first complete the panoramic image stitching of a single carriage, that is, to collect the image of a single carriage, then correct the distorted image after collection, and then convert the image into a bird's-eye view, and finally stitch the panoramic image of the single carriage. The whole vehicle image stitching is based on the positioning of the hinge point, that is, the coincidence point of the two carriages on the hinge plate, to calculate the positional relationship of the panoramic image, and complete the stitching of the panoramic image of the whole vehicle. In the process of stitching the image of a single carriage, only a simple stitching technology is used, and the image processing and image recognition depth are insufficient. When the vehicle is driving fast, image overlap and unstable pictures may occur; the whole vehicle panoramic image stitching is based on the positioning of the panoramic image position according to the hinge point, and does not consider the impact of the change in vehicle angle on the panoramic image when the vehicle turns, which will also affect the stitching effect of the whole vehicle panoramic image. Poor panoramic image effect of the whole vehicle will seriously affect the work efficiency of the crew and increase the risk of vehicle operation. Summary of the invention

[0004] The embodiments of the present application provide a panoramic image stitching method and related equipment for an electronically guided rubber-wheeled vehicle, which aims to solve the problem of poor stitching effect of existing panoramic images of the entire vehicle.

[0005] The present application provides a method for stitching panoramic images of an electronically guided rubber-wheeled vehicle, comprising the following steps:

[0006] The images to be spliced ​​are obtained by using preset cameras arranged on each carriage of the electronic guided rubber-wheeled vehicle, and the offset angle data between different carriages are obtained by using angle sensors arranged on each articulated plate of the electronic guided rubber-wheeled vehicle;

[0007] Acquire image features of the images to be stitched by using a generative adversarial network;

[0008] According to the offset angle data, the image features are spliced ​​according to a preset splicing rule to obtain a panoramic image of the electronically guided rubber-wheeled vehicle.

[0009] Optionally, before the step of acquiring the image features of the images to be stitched by using a generative adversarial network, the method further includes:

[0010] Input the preset random image into the generator in the initial adversarial generation network to obtain the generated sample;

[0011] According to the generated samples and the real image data, the generator loss of the generator is calculated by the discriminator in the initial adversarial generation network, and the generator parameters are updated according to the generator loss by back propagation to obtain the target generator;

[0012] Taking the generated sample as a negative example and the real image data as a positive example, calculating the discriminator loss of the discriminator;

[0013] Update the weight parameters of the discriminator by using the discriminator loss function and the discriminator loss to obtain a target discriminator;

[0014] A generative adversarial network that meets preset conditions is obtained according to the target generator and the target discriminator.

[0015] Optionally, the step of stitching the image features according to the offset angle data and a preset stitching rule to obtain a panoramic image of the electronically guided rubber-wheeled vehicle includes:

[0016] Determine a first spliced ​​carriage and a second spliced ​​carriage according to a splicing order in a preset splicing rule, and obtain a first offset angle between the first spliced ​​carriage and the second spliced ​​carriage;

[0017] Using the first offset angle as the first data in the offset angle data, and correcting the image features corresponding to the second spliced ​​carriage according to the first data to obtain a first target image to be spliced;

[0018] According to a preset stitching method in a preset stitching rule, stitching the first target image to be stitched and the image features corresponding to the first stitching carriage to obtain a first segment of the image;

[0019] Determine, according to the splicing order in the preset splicing rule, a third splicing carriage adjacent to the second splicing carriage and far away from the first splicing carriage;

[0020] Acquire a second offset angle between the third spliced ​​carriage and the second spliced ​​carriage, use the second offset angle as second data in the offset angle data, and correct the image features corresponding to the third spliced ​​carriage according to the second data to obtain a second target image to be spliced;

[0021] According to the preset stitching method, stitching the second target image to be stitched with the first segment of image to obtain a second segment of image;

[0022] The second segment of images is used as a panoramic image of the electronically guided rubber-wheeled vehicle.

[0023] Optionally, the step of stitching the first target image to be stitched and the image features corresponding to the first stitching carriage according to a preset stitching method in a preset stitching rule includes:

[0024] Acquire a first cut set of the first target image to be stitched and a second cut set of image features corresponding to the first stitched carriage, and acquire three-channel data of the first cut set and three-channel data of the second cut set;

[0025] The three-channel data of the first cut set and the three-channel data of the second cut set are matched. If the three-channel data of the first cut set and the three-channel data of the second cut set match, it is determined that the first cut set and the second cut set are an overlapping area, and splicing is performed according to the first cut set and the second cut set.

[0026] Optionally, the step of correcting the image features corresponding to the second spliced ​​carriage according to the first data to obtain the first target image to be spliced ​​includes:

[0027] Calculating, based on the first data, correction data of image features corresponding to the second spliced ​​carriage by using a preset view transformation matrix;

[0028] The image features corresponding to the second spliced ​​carriage are corrected based on the correction data to obtain a first target image to be spliced.

[0029] Optionally, the offset angle data includes: a horizontal offset angle and a vertical offset angle.

[0030] Optionally, the step of acquiring the images to be stitched by using preset cameras arranged on each carriage of the electronically guided rubber-wheeled vehicle includes:

[0031] At least two panoramic images of each carriage are obtained by using a preset camera arranged on each carriage of the electronic guided rubber-tyred vehicle;

[0032] The panoramic images of the same carriage are stitched together to obtain images to be stitched of each carriage.

[0033] In a second aspect, the present application provides a panoramic image stitching device for an electronically guided rubber-wheeled vehicle, the device comprising:

[0034] A data acquisition module, used to obtain images to be spliced ​​through preset cameras arranged on each carriage of the electronic guided rubber-wheeled vehicle, and to obtain offset angle data between different carriages through angle sensors arranged on each articulated plate of the electronic guided rubber-wheeled vehicle;

[0035] A feature extraction module, used for obtaining the image features of the image to be stitched through a generative adversarial network;

[0036] The splicing module is used to splice the image features according to the offset angle data and a preset splicing rule to obtain a panoramic image of the electronically guided rubber-wheeled vehicle.

[0037] In a third aspect, the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the computer program.

[0038] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0039] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the images to be spliced ​​are obtained by preset cameras arranged on each carriage of the electronic guided rubber-wheeled vehicle, and the offset angle data between different carriages are obtained by angle sensors arranged on each articulated plate of the electronic guided rubber-wheeled vehicle; the image features of the images to be spliced ​​are obtained by an adversarial generative network; according to the offset angle data, the image features are spliced ​​according to preset splicing rules to obtain a panoramic image of the electronic guided rubber-wheeled vehicle. The adversarial generative network is introduced to achieve more accurate and clear image processing and recognition; the lateral and longitudinal relative angle change information (offset angle data) of each two carriages is accurately obtained by the articulated plate angle sensor, so that the generated panoramic image is more in line with reality, and accurate and stable splicing of panoramic images of electronic guided rubber-wheeled vehicles with more than three trains is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1This is a flow chart of a panoramic image stitching method for an electronically guided rubber-wheeled vehicle provided by the present invention;

[0042] Figure 2 It is a flow chart of generator training in a panoramic image stitching method of an electronically guided rubber-wheeled vehicle provided by the present invention;

[0043] Figure 3 It is a discriminator training flow chart in a panoramic image stitching method of an electronically guided rubber-wheeled vehicle provided by the present invention;

[0044] Figure 4 This is an application scenario diagram of a panoramic image stitching method for an electronically guided rubber-wheeled vehicle provided by the present invention;

[0045] Figure 5 It is a schematic diagram of the structure of a panoramic image stitching device for an electronically guided rubber-wheeled vehicle provided in one embodiment of the present application;

[0046] Figure 6 is a structural block diagram of an electronic device in one embodiment;

[0047] Figure 7 FIG. 4 is a structural block diagram of a computer storage medium in another embodiment. DETAILED DESCRIPTION

[0048] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0049] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0050] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0051] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0052] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0053] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0054] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0055] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0056] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0057] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]" depending on the context.

[0058] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0059] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0060] like Figure 1 As shown, the embodiment of the present application provides a method for stitching panoramic images of an electronically guided rubber-wheeled vehicle, comprising the following steps:

[0061] S101, obtaining images to be spliced ​​by using preset cameras arranged on each carriage of the electronic guided rubber-wheeled vehicle, and obtaining offset angle data between different carriages by using angle sensors arranged on each articulated plate of the electronic guided rubber-wheeled vehicle;

[0062] Exemplarily, the electronically guided rubber-wheeled vehicle includes a head car, a middle car and a tail car. Three preset cameras are provided for the head car and the tail car, and two preset cameras are provided for the middle car. Specifically, a preset camera is provided on each of the left and right sides of the head car, a preset camera is provided on the front side of the head car, a preset camera is provided on each of the left and right sides of the middle car, a preset camera is provided on each of the left and right sides of the tail car, and a preset camera is provided on the rear side of the head car.

[0063] In a possible implementation manner, the offset angle data includes: a horizontal offset angle and a vertical offset angle.

[0064] Exemplarily, the angle sensors provided on each articulated plate of the electronically guided rubber-wheeled vehicle include a horizontal angle sensor and a vertical angle sensor.

[0065] S102, obtaining image features of the images to be stitched by using a generative adversarial network;

[0066] Exemplarily, the generative adversarial network is used to accurately identify image features. The generative adversarial network can improve model performance through competitive training, that is, ensure the accuracy of image features.

[0067] S103, stitching the image features according to the offset angle data and a preset stitching rule to obtain a panoramic image of the electronically guided rubber-wheeled vehicle.

[0068] The images to be spliced ​​are obtained by the preset cameras set on each carriage of the electronic guided rubber-wheeled vehicle, and the offset angle data between different carriages are obtained by the angle sensors set on each articulated plate of the electronic guided rubber-wheeled vehicle; the image features of the images to be spliced ​​are obtained by the adversarial generative network; according to the offset angle data, the image features are spliced ​​according to the preset splicing rules to obtain the panoramic image of the electronic guided rubber-wheeled vehicle. The adversarial generative network is introduced to achieve more accurate and clear image processing and recognition; the horizontal and vertical relative angle change information (offset angle data) of each two carriages is accurately obtained by the articulated plate angle sensor, so that the generated panoramic image is more in line with reality, and the accurate and stable splicing of the panoramic images of electronic guided rubber-wheeled vehicles with more than three groups is achieved.

[0069] In a possible implementation manner, before the step of obtaining the image features of the to-be-stitched images by using a generative adversarial network, the method further includes:

[0070] Input the preset random image into the generator in the initial adversarial generation network to obtain the generated sample;

[0071] According to the generated samples and the real image data, the generator loss of the generator is calculated by the discriminator in the initial adversarial generation network, and the generator parameters are updated according to the generator loss by back propagation to obtain the target generator;

[0072] Taking the generated sample as a negative example and the real image data as a positive example, calculating the discriminator loss of the discriminator;

[0073] Update the weight parameters of the discriminator by using the discriminator loss function and the discriminator loss to obtain a target discriminator;

[0074] A generative adversarial network that meets preset conditions is obtained according to the target generator and the target discriminator.

[0075] For example, Figure 2 As shown, the generator is trained using a discriminator, including: random image (preset random image) input; generator output is generated based on the random image; the discriminator determines whether the generator output is "real" or "fake"; the discriminator classification loss is calculated; back propagation is performed through the discriminator and the generator to obtain the gradient (generator loss); and the generator weight is changed according to the gradient.

[0076] For example, Figure 3 As shown, the learning and training of the discriminator includes:

[0077] This generator generates fake data instances that the discriminator uses as negative examples during training.

[0078] The discriminator classifies real environments, photos of people (real image data), and fake data generated by the generator (the generated samples); the discriminator loss penalizes the discriminator for misclassifying real images as fake images; the discriminator updates its weights by backpropagating through the discriminator loss function through the discriminator network.

[0079] In a possible implementation, the step of stitching the image features according to the offset angle data and a preset stitching rule to obtain a panoramic image of the electronically guided rubber-wheeled vehicle includes:

[0080] Determine a first spliced ​​carriage and a second spliced ​​carriage according to a splicing order in a preset splicing rule, and obtain a first offset angle between the first spliced ​​carriage and the second spliced ​​carriage;

[0081] Using the first offset angle as the first data in the offset angle data, and correcting the image features corresponding to the second spliced ​​carriage according to the first data to obtain a first target image to be spliced;

[0082] According to a preset stitching method in a preset stitching rule, stitching the first target image to be stitched and the image features corresponding to the first stitching carriage to obtain a first segment of the image;

[0083] Determine, according to the splicing order in the preset splicing rule, a third splicing carriage adjacent to the second splicing carriage and far away from the first splicing carriage;

[0084] Acquire a second offset angle between the third spliced ​​carriage and the second spliced ​​carriage, use the second offset angle as second data in the offset angle data, and correct the image features corresponding to the third spliced ​​carriage according to the second data to obtain a second target image to be spliced;

[0085] According to the preset stitching method, stitching the second target image to be stitched with the first segment of image to obtain a second segment of image;

[0086] The second segment of images is used as a panoramic image of the electronically guided rubber-wheeled vehicle.

[0087] For example, Figure 4 As shown in the figure, the relative position information of each carriage changes when the electronic guided rubber-tyred vehicle turns. The angle change of the carriage will cause the panoramic image of the carriage to change. When stitching the panoramic image of the whole vehicle, first complete the stitching of the panoramic images of the tail and middle carriages. According to the angle sensor of the articulated plate, the rotation angle P of the middle carriage relative to the tail carriage can be determined. According to the rotation angle of the carriage, the angle change of the panoramic image of this carriage is calculated, and the panoramic image stitching of the two carriages is completed in combination with the image features. Finally, the panoramic image stitching of the whole vehicle is performed, that is, the completed panoramic images of the tail and middle carriages are stitched with the panoramic image of the head carriage. The relative rotation angle of the middle and head carriages is Q, and the panoramic image stitching method is consistent with the stitching method of the panoramic images of the tail and middle carriages.

[0088] In a possible implementation, the step of stitching the first target image to be stitched and the image features corresponding to the first stitching carriage according to the preset stitching method in the preset stitching rule includes:

[0089] Acquire a first cut set of the first target image to be stitched and a second cut set of image features corresponding to the first stitched carriage, and acquire three-channel data of the first cut set and three-channel data of the second cut set;

[0090] The three-channel data of the first cut set and the three-channel data of the second cut set are matched. If the three-channel data of the first cut set and the three-channel data of the second cut set match, it is determined that the first cut set and the second cut set are an overlapping area, and splicing is performed according to the first cut set and the second cut set.

[0091] Exemplarily, the overlapping parts of the images captured by different preset cameras are determined by comparing the three-channel data of different cut sets.

[0092] In a possible implementation manner, the step of correcting the image features corresponding to the second spliced ​​carriage according to the first data to obtain the first target image to be spliced ​​includes:

[0093] Calculating, based on the first data, correction data of image features corresponding to the second spliced ​​carriage by using a preset view transformation matrix;

[0094] The image features corresponding to the second spliced ​​carriage are corrected based on the correction data to obtain a first target image to be spliced.

[0095] For example, in a three-dimensional coordinate system, the unit vector (1, 0, 0) on the X axis becomes (cosθ, sinθ, 0) after rotating around the Z axis by an angle θ, and the unit vector (0, 1, 0) on the Y axis becomes (-sinθ, cosθ, 0) after rotating around the Z axis by an angle θ. The unit vector on the Z axis does not change before and after the rotation and remains (0, 0, 1). In OpenGL, the model view transformation matrix is ​​a 4×4 matrix that follows the column vector priority matrix order. Therefore, the rotation matrix Rz(θ) composed of three unit vectors rotating around the Z axis with the origin as the center can be expressed as:

[0096]

[0097] That is, based on the first data, the correction data of the image features corresponding to the second spliced ​​carriage is calculated through a preset view transformation matrix.

[0098] In a possible implementation manner, the step of acquiring the images to be stitched by using a preset camera disposed on each carriage of the electronically guided rubber-wheeled vehicle includes:

[0099] At least two panoramic images of each carriage are obtained by using a preset camera arranged on each carriage of the electronic guided rubber-tyred vehicle;

[0100] The panoramic images of the same carriage are stitched together to obtain images to be stitched of each carriage.

[0101] Exemplarily, since any carriage on the electronically guided rubber-wheeled vehicle corresponds to at least two preset cameras, that is, each carriage has at least two panoramic images. To facilitate image stitching, the panoramic images of the same carriage are stitched first, and then the images to be stitched between different carriages are stitched.

[0102] In a possible implementation, Figure 5 As shown, the present application provides a panoramic image stitching device for an electronically guided rubber-wheeled vehicle, the device comprising:

[0103] The data acquisition module 201 is used to obtain the images to be spliced ​​through the preset cameras arranged on each carriage of the electronic guided rubber-wheeled vehicle, and to obtain the offset angle data between different carriages through the angle sensors arranged on each articulated plate of the electronic guided rubber-wheeled vehicle;

[0104] A feature extraction module 202, used for obtaining image features of the image to be stitched by using a generative adversarial network;

[0105] The stitching module 203 is used to stitch the image features according to the offset angle data and a preset stitching rule to obtain a panoramic image of the electronically guided rubber-wheeled vehicle.

[0106] Thirdly, Figure 6 As shown, the embodiment of the present application provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: obtaining images to be spliced ​​through a preset camera disposed on each carriage of the electronic guided rubber-wheeled vehicle, and obtaining offset angle data between different carriages through an angle sensor disposed on each articulated plate of the electronic guided rubber-wheeled vehicle;

[0107] Acquire image features of the images to be stitched by using a generative adversarial network;

[0108] According to the offset angle data, the image features are spliced ​​according to a preset splicing rule to obtain a panoramic image of the electronically guided rubber-wheeled vehicle.

[0109] Fourthly, Figure 7 As shown, the embodiment of the present application provides a computer-readable storage medium 400, wherein the computer-readable storage medium stores a computer program 411, and when the computer program is executed by a processor, the following steps are implemented: obtaining images to be spliced ​​through a preset camera arranged on each carriage of the electronic guided rubber-wheeled vehicle, and obtaining offset angle data between different carriages through an angle sensor arranged on each articulated plate of the electronic guided rubber-wheeled vehicle;

[0110] Acquire image features of the images to be stitched by using a generative adversarial network;

[0111] According to the offset angle data, the image features are spliced ​​according to a preset splicing rule to obtain a panoramic image of the electronically guided rubber-wheeled vehicle.

[0112] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0113] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0114] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0115] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment / terminal equipment and methods can be implemented in other ways. For example, the device / network equipment / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0116] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0117] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A panoramic image stitching method for an electronically guided rubber-wheeled vehicle, characterized in that: The following steps are involved: The images to be spliced ​​are obtained by using preset cameras arranged on each carriage of the electronic guided rubber-wheeled vehicle, and the offset angle data between different carriages are obtained by using angle sensors arranged on each articulated plate of the electronic guided rubber-wheeled vehicle; Acquire image features of the images to be stitched by using a generative adversarial network; According to the offset angle data, the image features are spliced ​​according to a preset splicing rule to obtain a panoramic image of the electronically guided rubber-wheeled vehicle.

2. The method for stitching panoramic images of an electronically guided rubber-wheeled vehicle according to claim 1, characterized in that: Before the step of obtaining the image features of the images to be spliced ​​by using a generative adversarial network, the method further includes: Input the preset random image into the generator in the initial adversarial generation network to obtain the generated sample; According to the generated samples and the real image data, the generator loss of the generator is calculated by the discriminator in the initial adversarial generation network, and the generator parameters are updated according to the generator loss by back propagation to obtain the target generator; Taking the generated sample as a negative example and the real image data as a positive example, calculating the discriminator loss of the discriminator; Update the weight parameters of the discriminator by using the discriminator loss function and the discriminator loss to obtain a target discriminator; A generative adversarial network that meets preset conditions is obtained according to the target generator and the target discriminator.

3. The method for stitching panoramic images of an electronically guided rubber-wheeled vehicle according to claim 1, characterized in that: The step of stitching the image features according to the offset angle data and a preset stitching rule to obtain a panoramic image of the electronically guided rubber-wheeled vehicle comprises: Determine a first spliced ​​carriage and a second spliced ​​carriage according to a splicing order in a preset splicing rule, and obtain a first offset angle between the first spliced ​​carriage and the second spliced ​​carriage; Using the first offset angle as the first data in the offset angle data, and correcting the image features corresponding to the second spliced ​​carriage according to the first data to obtain a first target image to be spliced; According to a preset stitching method in a preset stitching rule, stitching the first target image to be stitched and the image features corresponding to the first stitching carriage to obtain a first segment of the image; Determine, according to the splicing order in the preset splicing rule, a third splicing carriage adjacent to the second splicing carriage and far away from the first splicing carriage; Acquire a second offset angle between the third spliced ​​carriage and the second spliced ​​carriage, use the second offset angle as second data in the offset angle data, and correct the image features corresponding to the third spliced ​​carriage according to the second data to obtain a second target image to be spliced; According to the preset stitching method, stitching the second target image to be stitched with the first segment of image to obtain a second segment of image; The second segment of images is used as a panoramic image of the electronically guided rubber-wheeled vehicle.

4. The method for stitching panoramic images of an electronically guided rubber-wheeled vehicle as claimed in claim 3, characterized in that: The step of stitching the first target image to be stitched and the image features corresponding to the first stitching carriage according to the preset stitching method in the preset stitching rule includes: Acquire a first cut set of the first target image to be stitched and a second cut set of image features corresponding to the first stitched carriage, and acquire three-channel data of the first cut set and three-channel data of the second cut set; The three-channel data of the first cut set and the three-channel data of the second cut set are matched. If the three-channel data of the first cut set and the three-channel data of the second cut set match, it is determined that the first cut set and the second cut set are an overlapping area, and splicing is performed according to the first cut set and the second cut set.

5. The method for stitching panoramic images of an electronically guided rubber-wheeled vehicle as claimed in claim 3, characterized in that: The step of correcting the image features corresponding to the second spliced ​​carriage according to the first data to obtain the first target image to be spliced ​​includes: According to the first data, the correction data of the image features corresponding to the second spliced ​​carriage is calculated by a preset view transformation matrix; The image features corresponding to the second spliced ​​carriage are corrected based on the correction data to obtain a first target image to be spliced.

6. The method for stitching panoramic images of an electronically guided rubber-wheeled vehicle according to claim 1, characterized in that: The offset angle data includes: a horizontal offset angle and a vertical offset angle.

7. The method for stitching panoramic images of an electronically guided rubber-wheeled vehicle according to claim 1, characterized in that: The step of obtaining the images to be spliced ​​by using preset cameras arranged on each carriage of the electronic guided rubber-wheeled vehicle comprises: At least two panoramic images of each carriage are obtained by using a preset camera arranged on each carriage of the electronic guided rubber-tyred vehicle; The panoramic images of the same carriage are stitched together to obtain images to be stitched of each carriage.

8. An electronically guided rubber-wheeled vehicle panoramic image stitching device, characterized in that: The device comprises: A data acquisition module, used to obtain images to be spliced ​​through preset cameras arranged on each carriage of the electronic guided rubber-wheeled vehicle, and to obtain offset angle data between different carriages through angle sensors arranged on each articulated plate of the electronic guided rubber-wheeled vehicle; A feature extraction module, used for obtaining the image features of the image to be stitched through a generative adversarial network; The splicing module is used to splice the image features according to the offset angle data and a preset splicing rule to obtain a panoramic image of the electronically guided rubber-wheeled vehicle.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images