Vehicle-related payment method, vehicle-related payment system and storage medium for highway traffic

Abstract

This application relates to payment system technology, and discloses a vehicle-related payment method, a vehicle-related payment system and a storage medium for passing through a highway, including: based on a camera deployed at the highway entrance, collecting a vehicle image of a vehicle to pass through; using a pre-trained artificial intelligence model to identify the license plate number; when it is detected that there is an associated vehicle in the vehicle-related payment system for the license plate number, the vehicle-related payment system notifies the railing device at the highway entrance to raise the bar for release; when a camera deployed at the highway exit collects a vehicle image of a vehicle to pass through and recognizes the corresponding license plate number, the vehicle-related payment system calls a charging model to calculate the highway toll; according to the payment method pre-signed by the vehicle associated with the license plate number in the vehicle-related payment system, automatically settle the highway toll and notify the railing device at the highway exit to raise the bar for release. This application aims to provide a vehicle-related payment system for a highway based on license plate recognition to improve the passing efficiency of vehicles at highway entrances and exits.

Description

Technical Field

[0001] The present application relates to the technical field of payment systems, and particularly to a vehicle-related payment method, a vehicle-related payment system, and a computer-readable storage medium for highway passage. Background Art

[0002] Currently, the automatic passage mode of vehicles at highway entrances and exits mainly uses ETC (Electronic Toll Collection System) for billing. This requires vehicle owners to install ETC on-vehicle devices in advance. For the majority of vehicle owners, the purchase and maintenance of ETC on-vehicle devices are additional expenses. Especially for those vehicle owners who rarely use the highway, the usage frequency of ETC on-vehicle devices is low, and the cost performance of installing the devices is not high, making the cost of highway ETC passage particularly prominent.

[0003] With the development of image recognition technology, vehicle-related payment scenarios using license plate image recognition are becoming more and more widely used in life, especially in parking lot scenarios. However, the existing license plate image recognition technology still faces many challenges in highway scenarios. On the one hand, the driving speed of vehicles on the road is often higher, and motion blur is more likely to occur during image acquisition. On the other hand, the lighting conditions on highways are complex and changeable, which brings great interference to license plate image recognition. In some periods, the license plate may be in direct sunlight or shadow areas (direct sunlight will cause the license plate to reflect light, resulting in overexposed images and loss of character information; while in shadow areas, the image will be darker and the contrast will decrease), making it difficult for the existing license plate image recognition technology to meet the accurate recognition requirements in complex highway scenarios, and it is also difficult to support the automatic passage of vehicles at highway entrances and exits as stably and efficiently as the ETC system, thus restricting its wide application in the field of highway vehicle-related payment.

[0004] The above content is only used to assist in understanding the technical solution of the present application, and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of the present application is to provide a vehicle-related payment method, a vehicle-related payment system, and a computer-readable storage medium for highway passage, aiming to provide a highway vehicle-related payment system based on license plate recognition to improve the efficiency of vehicle passage at highway entrances and exits.

[0006] To achieve the above purpose, the present application provides a vehicle-related payment method for highway passage, including the following steps:

[0007] Based on the camera deployed at the highway entrance, collect the vehicle image of the vehicle to be passed;

[0008] Using a pre-trained artificial intelligence model to identify the license plate number corresponding to a vehicle image; wherein, the artificial intelligence model adopts a dual-branch network, one branch is used to estimate the motion blur parameters of the image, and the other branch is used to estimate the weights of different spectral channels of the image, and an interaction mechanism is introduced between the outputs of the two branches to enable the estimated outputs of each branch to act on the feature map adjustment of other branches; and, in each branch, the original image is processed respectively by using the estimated output of each branch and the adjusted feature map, and the images processed by each branch are fused to generate a fused image after deblurring and spectral optimization for license plate number recognition;

[0009] When it is detected that the identified license plate number has an associated vehicle in the vehicle-related payment system, the vehicle-related payment system notifies the railing device at the highway entrance to lift the bar and release the vehicle, and records the corresponding passing information;

[0010] When a camera deployed at the highway exit captures a vehicle image of a vehicle to pass through, and the license plate number is identified by using the artificial intelligence model, the vehicle-related payment system calls the billing model provided by the highway management system, and calculates the highway toll according to the passing information of the current identified license plate number at the highway entrance and the passing information at the current highway exit; wherein, if a camera deployed at the highway section toll collection point captures the corresponding vehicle image and the license plate number is identified by using the artificial intelligence model, the passing information of the highway section toll collection point associated with the license plate number is used as one of the calculation factors for the highway toll;

[0011] Automatically settle the highway toll according to the payment method pre-signed by the vehicle associated with the license plate number in the vehicle-related payment system;

[0012] After the highway toll settlement is successful, the vehicle-related payment system notifies the railing device at the highway exit to lift the bar and release the vehicle.

[0013] To achieve the above object, the present application also provides a vehicle-related payment system, the vehicle-related payment system includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the computer program is executed by the processor, the steps of the vehicle-related payment method for passing through the highway as described above are implemented.

[0014] To achieve the above object, the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the vehicle-related payment method for passing through the highway as described above are implemented.

[0015] The vehicle-related payment method, vehicle-related payment system, and computer-readable storage medium provided by this application adopt a special dual-branch network pre-trained artificial intelligence model. Even if the vehicle is moving at high speed, resulting in blurred images, or the lighting conditions on the highway are complex, it can generate clear and accurate fused images for license plate recognition, meeting the accurate recognition requirements in complex highway scenarios. Moreover, once the recognition is successful, the vehicle-related payment system will respond quickly and automatically settle the fees and release the vehicle according to the payment method pre-signed by the vehicle, without manual intervention, reducing the vehicle waiting time and providing an efficient and convenient high-speed passage payment experience for vehicle owners. Furthermore, throughout the vehicle-related payment and passage process, vehicle owners do not need to pre-install devices such as ETC for highway toll billing, reducing the cost of vehicle passage on the highway. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the steps of the vehicle-related payment method for passing on the highway in an embodiment of this application;

[0017] Figure 2 It is a schematic diagram of the architecture of the dual-branch network of the artificial intelligence model in an embodiment of this application;

[0018] Figure 3 It is a schematic diagram of the internal architecture of the vehicle-related payment system in an embodiment of this application.

[0019] The implementation, functional features, and advantages of the objectives of this application will be further described in conjunction with the embodiments with reference to the accompanying drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] The embodiments of this application will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain this application and should not be construed as limiting this application. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of this application.

[0021] In addition, if the descriptions in this application involve "first", "second", etc., they are only for descriptive purposes (such as for distinguishing the same or similar features), and should not be construed as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments may be combined with each other, but it must be based on the ability of those of ordinary skill in the art to implement. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0022] Refer to Figure 1 , in one embodiment, the vehicle-related payment method for highway passage includes:

[0023] Step S10: Based on the camera deployed at the highway entrance, collect the vehicle image of the vehicle to pass;

[0024] Step S20: Use a pre-trained artificial intelligence model to identify the license plate number corresponding to the vehicle image; wherein, the artificial intelligence model adopts a dual-branch network, one branch is used to estimate the motion blur parameter of the image, and the other branch is used to estimate the weights of different spectral channels of the image, and an interaction mechanism is introduced between the outputs of the two branches to make the estimated output of each branch act on the feature map adjustment of other branches; and, in each branch, respectively use the estimated output of each branch and the adjusted feature map to process the original image, and fuse the images processed by each branch to generate a de-blurred and spectrum-optimized fused image for license plate number recognition;

[0025] Step S30: When it is detected that the identified license plate number has an associated vehicle in the vehicle-related payment system, the vehicle-related payment system notifies the railing device at the highway entrance to lift the bar for release and records the corresponding passage information;

[0026] Step S40: When the camera deployed at the highway exit collects the vehicle image of the vehicle to pass and uses the artificial intelligence model to identify the corresponding license plate number, the vehicle-related payment system calls the billing model provided by the highway management system, and calculates the highway passage fee according to the passage information of the current identified license plate number at the highway entrance and the passage information at the current highway exit; wherein, if the camera deployed at the highway section billing point collects the corresponding vehicle image and uses the artificial intelligence model to identify the corresponding license plate number, the passage information of the highway section billing point associated with the license plate number is used as one of the calculation factors for the highway passage fee;

[0027] Step S50: Automatically settle the highway passage fee according to the payment method pre-signed by the vehicle associated with the license plate number in the vehicle-related payment system;

[0028] Step S60: After the highway passage fee is successfully settled, the vehicle-related payment system notifies the railing device at the highway exit to lift the bar for release.

[0029] In this embodiment, the terminal system for implementing the embodiment can be a vehicle-related payment system, or other devices or apparatuses (such as a control device) that control the vehicle-related payment system.

[0030] As described in step S10, cameras are usually pre-deployed at highway entrances, exits, and interval toll collection points. The system can establish communication connections with these existing highway cameras. In this way, the existing hardware device resources can be fully utilized, and the cost of accessing the existing system can be greatly reduced.

[0031] Optionally, the cameras are usually installed at appropriate positions above or on the side of the highway entrance and exit lanes. When installed above, it can capture vehicle images including license plates at an angle diagonally above the vehicle's front. When installed on the side, it can capture vehicle images including license plates from the diagonal side of the vehicle's front.

[0032] Optionally, vehicle sensors such as inductive loops or infrared sensors are installed on the highway entrance lane. When a vehicle enters the sensing area, the sensor will detect the presence of the vehicle and immediately send a trigger signal to the camera to start the image acquisition program. The camera can also acquire images in the form of a continuous video stream, and then use image processing algorithms to detect the entry of the vehicle in real time. Once a vehicle is detected entering the shooting range, an image frame containing the vehicle is intercepted from the video stream.

[0033] As described in step S20, the vehicle images collected by the cameras at the highway entrance are processed using a pre-trained artificial intelligence model to accurately identify the corresponding license plate numbers. To address the problems of image motion blur that may occur during vehicle driving and spectral differences under different lighting conditions, refer to Figure 2 , this artificial intelligence model adopts a unique dual-branch network architecture, where one branch is the motion blur estimation branch and the other branch is the spectral channel estimation branch.

[0034] During vehicle driving, due to its moving state, the camera is likely to produce motion blur when capturing images. The main task of the motion blur estimation branch is to analyze the vehicle images and estimate the motion blur parameters caused by motion in the images. These parameters include the direction of blur (such as horizontal, vertical, or inclined direction) and the degree of blur (i.e., the length of blur). The motion blur estimation branch will perform in-depth analysis on the local features of the image (such as the sharpness of edges, the continuity of textures, etc.), and determine the motion blur parameters of the current image by comparing with the image features of different blur degrees and directions stored in the pre-trained model.

[0035] Different lighting conditions (such as sunny days, cloudy days, backlighting, etc.) will cause differences in the performance of images in various spectral channels (such as red, green, and blue channels). The role of the spectral channel estimation branch is to estimate the weights of different spectral channels of the image to optimize the display effect of the image under different spectra. The spectral channel estimation branch will analyze features such as the color distribution and brightness of the image, and according to the weight allocation rules of spectral channels under different lighting conditions learned in the pre-trained model, allocate appropriate weights to each spectral channel of the current image.

[0036] To enable better fusion of the information of the two branches and improve the image processing effect, an interaction mechanism is introduced between the outputs of the two branches. This mechanism makes the estimated outputs of the two branches not independent of each other, but can influence and adjust each other.

[0037] Optionally, the output of the motion blur estimation branch acts on the feature map of the spectral channel estimation branch to adjust it. For example, if the motion blur parameter indicates that the image has strong blur in a certain direction, then when adjusting the spectral channel weights, the influence of this blur on color perception will be considered, and the feature map will be corrected accordingly. Conversely, the output of the spectral channel estimation branch will also act on the feature map of the motion blur estimation branch in the same way.

[0038] Optionally, the artificial intelligence model encodes the estimated output of the motion blur estimation branch to generate a weight matrix, and performs weighted fusion on the weight matrix and the first feature map of the spectral channel estimation branch to obtain an adjusted first feature map.

[0039] Optionally, the artificial intelligence model performs a dot product operation on the estimated output of the spectral channel estimation branch and the second feature map of the motion blur estimation branch to obtain a tensor with the same shape as the second feature map;

[0040] The tensor is normalized to obtain attention weights;

[0041] The attention weights are multiplied element-wise with the second feature map in the spatial dimension to obtain an adjusted second feature map.

[0042] Optionally, based on the adjusted second feature map and combined with the previously estimated motion blur parameters, the original image is deblurred to attempt to recover the detailed information lost by the image due to motion blur and spectral effects, making key information such as license plates in the image clearer. The following is the generation process of the clear image after deblurring:

[0043] Construct a motion blur kernel based on the motion blur parameters (blur direction and blur length) output by the motion blur estimation branch. The blur kernel is a two-dimensional matrix, whose size is mainly determined by the blur length, and the shape is related to the blur direction. During the construction process, first create a zero matrix of a specific size, then set non-zero elements on the matrix according to the blur direction and length, and finally normalize the matrix to ensure that the sum of all elements is 1. This blur kernel simulates the process of image motion blur and is a key tool for subsequent deblurring operations.

[0044] Then, Wiener filtering is adopted to comprehensively consider the blur kernel and the noise situation in the image, and the original clear image is restored by minimizing the mean square error. During processing, the adjusted second feature map is used as a kind of guiding information and incorporated into the filtering process. That is, according to the feature intensity in different regions of the second feature map, the filtering parameters are dynamically adjusted, so that a more refined deblurring operation is performed in regions with obvious features, while a relatively conservative filtering method is adopted in regions with weaker features.

[0045] Specifically, estimate the noise situation in the original image F0 to obtain the noise power spectrum P n (u, v); perform a two-dimensional discrete Fourier transform (DFT) on the original image F0 to obtain its frequency domain representation G(u, v); similarly, perform a two-dimensional discrete Fourier transform on the blur kernel to obtain H(u, v); perform a two-dimensional discrete Fourier transform on the adjusted second feature map F2 to obtain F2'(u, v). Then, calculate the Wiener filtering function, and the calculation formula of the Wiener filtering function is:

[0046] ;

[0047] where, H * (u, v) is the conjugate complex number of H(u, v); P f (u, v) is the power spectrum of the original clear image. Since the original clear image is unknown, the model will make a reasonable estimate according to the prior knowledge obtained through pre-training. That is, during the training stage of the model, a large number of clear images and corresponding blurred images are used to train the model. Through learning these data, the model can understand the power distribution laws of different types of images at different frequencies, as well as the influence of image blur and noise on the power spectrum. For example, the model can find similar image samples from the training data according to some features of the blurred image (such as average brightness, contrast, texture complexity, etc.), and then use the power spectra of these samples as estimates of the power spectrum of the original clear image of the current blurred image. Through this prior knowledge-based estimation method, the model can reasonably estimate P f (u, v) when the original clear image is unknown, so as to achieve effective Wiener filtering deblurring processing.

[0048] In order to dynamically adjust the filtering parameters according to the feature intensities in different regions of the second feature map, an adjustment factor α(u, v) is predefined, and α(u, v) is calculated according to the amplitude of F2'(u, v):

[0049] ;

[0050] The adjustment factor is incorporated into the Wiener filtering function to obtain the adjusted Wiener filtering function:

[0051] ;

[0052] where k > 1 is a constant used to control a relatively conservative filtering method in regions with weak features, and its specific value can also be determined based on the prior knowledge pre-learned by the model. When α(u, v) approaches 1 (in regions with obvious features), the filtering function approaches the original Wiener filtering function for a more refined deblurring operation; when α(u, v) approaches 0 (in regions with weak features), the denominator in the filtering function increases, and the filtering effect is relatively conservative.

[0053] Multiply the adjusted Wiener filtering function w2(u, v) by the frequency-domain representation G(u, v) of the original image to obtain the frequency-domain representation F0'(u, v) of the restored image. Then perform a two-dimensional inverse discrete Fourier transform (IDFT) on F0'(u, v) to obtain the restored clear image.

[0054] It should be noted that due to factors such as the aberration of the optical system and atmospheric scattering, images in different spectral channels may have different degrees of blurring. Some blurring phenomena may be directly related to spectral characteristics (such as chromatic aberration blurring). Chromatic aberration will cause the images in different spectral channels to be displaced spatially, making the images blurred. By adjusting the spectral channel weights and combining with the deblurring algorithm, this spectral-related blurring can be compensated, so as to more accurately restore the clear details of the image.

[0055] When using deblurring algorithms such as Wiener filtering, the spectral channel weights can be used as a guiding information to dynamically adjust the filtering parameters, so as to perform a more refined deblurring operation in spectral channels with obvious features, and adopt a relatively conservative filtering method in channels with weak features, thereby improving the deblurring effect.

[0056] After obtaining the adjusted first feature map, the original image can be processed according to the following scheme using the estimated output of the spectral channel estimation branch and the first feature map to obtain the spectral optimization map after removing the influence of motion blur:

[0057] Since the spectral channel estimation branch has analyzed features such as the color distribution and brightness of the image, according to the weight assignment rules of spectral channels under different lighting conditions learned in the pre-trained model, appropriate weights can be assigned to each spectral channel (such as the red, green, and blue channels) of the current image. Let the output of the spectral channel estimation branch be W s =[W r ,W g ,W b , where W r , W g , W b are the weights of the red, green, and blue channels respectively.

[0058] The adjusted first feature map F1 contains the information after the motion blur parameter adjusts the feature map of the spectral channel estimation branch. The adjusted first feature map F1 can be combined with the spectral channel weight W s to highlight or suppress the features of different spectral channels. Assume that F1 is split into feature maps F r , F g , F b of the three color channels. Then the weighted feature maps of the three color channels are respectively F r ' = W r × F r , F g ' = W g × F g , F b ' = W b × F b .

[0059] Assume that the original image F0 is also split into images I r , I g , I b of the three color channels. Apply the weighted feature maps of the three color channels to the corresponding channels of the original image to obtain the adjusted channel images I r ' = I r + β × F r ', I g ' = I g + β × F g ', I b ' = I b + β × F b ', where β is an adjustment coefficient used to control the influence degree of the weighted feature map on the original image, and the optimal value of β can be learned in advance during the model training process.

[0060] Combine the adjusted three channel images I r ', I g ', I b'Re - merge into a complete image, and then a spectral optimization map with the influence of motion blur weakened can be obtained.

[0061] After obtaining the de - blurred clear image processed by the motion blur estimation branch and the spectral optimization map processed by the spectral channel estimation branch, the images obtained from the two branches are then fused. The purpose of fusion is to combine the advantages of the two - branch processing to generate a fused image that not only removes motion blur but also optimizes the spectrum.

[0062] Among them, the weighted average method can be adopted to sum the values of corresponding pixel points according to the weights to obtain the pixel values of the fused image.

[0063] Since motion blur has a greater impact on the recognition accuracy of license plate numbers than the optimization of spectral channel weights, the first weight of the motion blur estimation branch is set to be greater than the second weight of the spectral channel estimation branch (and the sum of the first weight and the second weight is 1). On this premise, the corresponding weights can also be adjusted according to the credibility of the processing results of the two branches (this can use image quality evaluation metrics (such as peak signal - to - noise ratio PSNR, structural similarity index SSIM, etc.) to evaluate the credibility, and the higher the credibility, the greater the weight) to evaluate the quality of the two images, and the weights are dynamically allocated according to the evaluation results (the final allocation result still needs to satisfy that the first weight is greater than the second weight, and their sum is 1).

[0064] Conventional processing for removing motion blur and spectral optimization of license plate images is often independent, such as de - blurring first and then spectral optimization, or spectral optimization first and then de - blurring. This approach ignores the mutual influence between removing motion blur and spectral optimization. If de - blurring is performed first, during this process, the blurred image may make the color and illumination information inaccurate. When performing spectral optimization later, since the previous de - blurring may have damaged certain image features to some extent, it is difficult for spectral optimization to be adjusted based on accurate color and illumination features, resulting in the color of the optimized image being unnatural and having problems such as color cast; conversely, if spectral optimization is performed first, the blurred image will make it impossible to accurately judge the true color distribution and illumination conditions during the optimization process, making the spectral optimization may be excessive or insufficient. Then, when performing de - blurring, due to the feature changes caused by the previous spectral optimization, it will affect the recognition of image features such as edges and textures by the de - blurring algorithm, thereby resulting in poor de - blurring effect, and the image still has a blurred feeling or artifacts.

[0065] The interaction mechanism introduced by the dual-branch network adopted in the artificial intelligence model of this application enables the motion blur estimation branch and the spectral channel estimation branch to share the feature information extracted by each other, thus avoiding the occurrence of the above problems. For example, features such as image edges and textures extracted by the motion blur estimation branch can provide references for spectral channel weight estimation, helping it better analyze the color distribution and lighting conditions of the image; while the color features extracted by the spectral channel estimation branch can also assist in motion blur parameter estimation, improving the judgment accuracy of the blur direction and degree.

[0066] After generating the deblurred and spectrally optimized fused image, a deep learning-based character recognition algorithm can be used to recognize the fused image to identify the corresponding license plate number. This algorithm will locate and segment the license plate area in the fused image, segment the characters on the license plate into individual characters, and then classify and recognize each character to finally obtain the complete license plate number.

[0067] As described in step S30, when the license plate number corresponding to the vehicle image is successfully recognized using the pre-trained artificial intelligence model, the vehicle-related payment system will immediately compare the recognized license plate number with the associated information stored internally in the system. The vehicle-related payment system stores license plate number information of a series of contracted vehicles (including vehicles that have contracted for online payment and vehicles that have contracted for prepaying tolls (i.e., prepaid vehicles)). The owners of these vehicles have reached a cooperation agreement with the vehicle-related payment system in advance and entered their vehicle information into the system.

[0068] If it is detected that the recognized license plate number has an associated vehicle in the vehicle-related payment system, this means that the owner of the vehicle has completed the necessary contracting process and meets the payment conditions for highway passage. At this time, the vehicle-related payment system will quickly send a bar-lifting instruction to the bar device at the highway entrance. After receiving the instruction, the bar device will immediately execute the bar-lifting action to allow the vehicle waiting to pass to enter the highway smoothly.

[0069] While notifying the bar device to lift the bar and let the vehicle pass, the vehicle-related payment system will record the vehicle's passage information in detail. These passage information includes but is not limited to the vehicle's license plate number, the time of entering the highway, the specific entrance location, etc. These recorded information will be stored in the database of the vehicle-related payment system as an important basis for subsequent calculation of highway tolls and querying of the vehicle's passage history.

[0070] If it is detected that the recognized license plate number has no associated vehicle in the vehicle-related payment system, then the vehicle may not be signed up with the vehicle-related payment system. In this case, the vehicle-related payment system may trigger a corresponding prompt mechanism, such as displaying a prompt message on the display screen at the highway entrance to guide the vehicle owner to perform the signing-up operation; or the railing device will not lift to let the vehicle pass, and the vehicle owner is required to use other traditional access methods (such as physical card access, ETC access, etc.) to obtain the access permission.

[0071] As described in step S40, when the vehicle reaches the highway exit, the cameras installed at specific positions at the exit will immediately start working. These cameras are carefully deployed to ensure that they can comprehensively and clearly capture the images of the vehicle to be passed, especially the license plate part.

[0072] After the vehicle image is collected, the system immediately uses the license plate recognition method described in step S20, calls the pre-trained artificial intelligence model, and performs license plate recognition on the vehicle image.

[0073] After the vehicle-related payment system successfully recognizes the license plate number of the vehicle at the highway exit, it will quickly search for the access information at the highway entrance corresponding to the license plate number in the system database. These entrance access information has been accurately recorded when the vehicle enters the highway, including the specific time when the vehicle enters the highway, accurate to the minute, second, and specific entrance location, such as the name or number of a specific toll station.

[0074] At the same time, the vehicle-related payment system will call the billing model of the highway management system. This billing model is carefully designed by relevant engineers of the highway management system through long-term research and practice, taking into account various factors. It is based on a large amount of traffic data, cost analysis, and policy regulations, aiming to scientifically and reasonably calculate the highway toll of the vehicle.

[0075] The vehicle-related payment system will integrate the previously recorded entrance access information and the currently obtained exit access information. The exit access information also includes the accurate departure time and the specific exit location. These information are the basic elements for calculating the toll, providing a key basis for determining the driving route and time range of the vehicle.

[0076] In addition, at some specific sections of the highway, highway interval toll points are set. These toll points are also equipped with cameras with the same functions as those at the entrance and exit. When the vehicle passes through these toll points, the cameras will automatically collect the vehicle image and use the same artificial intelligence model to recognize the license plate number. If the image of the corresponding vehicle is collected at the highway interval toll point and the license plate number is successfully recognized, the access information generated by the vehicle at the highway interval toll point at present (these information include the specific time when the vehicle passes through the toll point and the accurate location of the toll point) will be associated with the corresponding license plate number.

[0077] When the vehicle-related payment system calls the billing model to calculate the highway toll of a vehicle, if corresponding passing information is generated at the highway section toll points based on the license plate number query, the vehicle-related payment system will obtain the passing information of the vehicle at these highway section toll points and incorporate it as a calculation factor into the calculation of the highway toll.

[0078] Then, the called billing model will combine the entry and exit passing information of the vehicle, comprehensively consider factors such as driving mileage, driving time, and toll standards for different sections (such as highway section toll points), and accurately calculate the highway toll of the vehicle. For example, if the vehicle passes through multiple highway section toll points, the billing model will calculate the toll for each section according to the toll standard of each section and then accumulate the total toll.

[0079] As described in step S50, when the vehicle-related payment system accurately calculates the highway toll of the vehicle, the system will immediately perform precise information matching and retrieval operations in its huge database based on the identified license plate number. A large amount of vehicle-related information is stored in the database, and each license plate number corresponds to a detailed file, which contains key information such as the payment method pre-signed by the vehicle and the vehicle-related payment system. The system can quickly locate the record corresponding to the license plate number in a short time using an efficient indexing algorithm.

[0080] Once the corresponding record is found, the system will confirm the payment method pre-signed by the vehicle and automatically settle the corresponding highway toll according to the pre-signed payment method.

[0081] Among them, the pre-signed payment method can be online payment (such as using digital currency, third-party payment platforms, bank card binding, etc.), prepayment (paying a certain amount in advance), etc.

[0082] As described in step S60, if the vehicle-related payment system successfully automatically settles the corresponding highway toll, it will send a bar-lifting instruction to the bar device at the highway exit. After receiving the bar-lifting instruction, the control module of the bar device will drive the motor or other actuators to lift the bar. At the same time, the bar device will feedback an execution result information to the vehicle-related payment system, indicating that the bar-lifting operation has been completed.

[0083] After receiving the execution result feedback from the bar device, the vehicle-related payment system confirms that the vehicle has been released. The system will record the release time of the vehicle and relevant operation information, completing the entire highway toll settlement and release process. At the same time, these records will be stored and analyzed as important business data for subsequent operation management and statistical report generation.

[0084] If the vehicle-related payment system fails to automatically settle the corresponding highway toll successfully, corresponding prompt messages shall be output through the display screen at the highway exit or the associated devices of the vehicle (such as in-vehicle devices, mobile devices, etc.) according to the reasons for the settlement failure to prompt the vehicle owner to make up the amount or settle manually, etc.

[0085] In one embodiment, a special dual-branch network is adopted to pre-train an artificial intelligence model. Even if the vehicle is driving at high speed, resulting in blurred image motion, or the lighting conditions on the highway are complex, a clear and accurate fused image can be generated for license plate recognition, meeting the accurate recognition requirements in complex highway scenarios. Moreover, once the recognition is successful, the vehicle-related payment system will respond quickly, automatically settle the fees according to the payment method pre-signed by the vehicle, and release the vehicle without manual intervention, reducing the vehicle waiting time and providing the vehicle owner with an efficient and convenient highway toll payment experience. Moreover, throughout the vehicle-related payment and passing process, the vehicle owner does not need to pre-install devices such as ETC for highway toll billing, reducing the cost of vehicle passing on the highway. Especially for vehicle owners with low highway passing frequencies, it is more economical and affordable.

[0086] In one embodiment, on the basis of the above embodiment, when initially performing Wiener filtering in step S20, since the original clear image is unknown, the power spectrum P f (u, v) of the original clear image used for performing Wiener filtering is estimated based on the prior knowledge pre-trained by the model, and there may be certain errors in this estimation. The restored clear image obtained by subsequent processing is obtained after one deblurring process, and its power spectrum can more accurately reflect the characteristic distribution of the original image in the frequency domain.

[0087] Therefore, after obtaining a clear image by performing one deblurring process on the original image, the power spectrum of the clear image can be used to update the power spectrum P f (u, v) of the original clear image in the Wiener filtering functions w1(u, v) and w2(u, v), and on this basis, re-perform the deblurring process on the original image to further optimize the deblurring effect and obtain a clear image with better quality.

[0088] However, re-performing Wiener filtering requires a series of calculations again, inevitably increasing the computational complexity. Therefore, in order to avoid the problems that may be faced when using the power spectrum of the restored clear image to replace the original P f (u, v) for re-filtering, the power spectrum of the finally generated clear image can be compared with the P f (u, v) used by the original Wiener filtering function first to determine whether it is necessary to update the power spectrum and re-generate the clear image.

[0089] Optionally, after obtaining the restored clear image, perform a two-dimensional discrete Fourier transform (DFT) on it to transform the image from the spatial domain to the frequency domain. Let the restored clear image be I rec , and after DFT, obtain its frequency domain representation I rec (u, v), and then calculate its power spectrum:

[0090] .

[0091] To effectively compare P rec (u, v) and the original P f (u, v), the mean and standard deviation of the two can be calculated respectively, and then the difference in means between the two is compared with the first threshold (a preset mean difference threshold), and the difference in standard deviations between the two is compared with the second threshold (a preset standard deviation difference threshold). Among them, if the difference in means between the two is greater than the first threshold, it indicates that the difference in means between the two is significant; if the difference in standard deviations between the two is greater than the second threshold, it is considered that the difference in dispersion between the two is significant.

[0092] Optionally, if both indicators indicate that P rec (u, v) and P f (u, v) are significantly different, it means that the power spectrum of the restored clear image is quite different from the originally estimated power spectrum. Therefore, using P rec (u, v) to replace P f (u, v) to re-perform Wiener filtering can largely further improve the deblurring effect, and it can be considered to regenerate the clear image. If at least one indicator indicates that P rec (u, v) and P f (u, v) are not significantly different, it means that the originally estimated P f (u, v) can already better reflect the frequency domain characteristics of the image, and re-filtering may not bring obvious improvement and may even introduce new problems. In this case, it is not recommended to use P rec (u, v) to replace P f (u, v) to re-perform Wiener filtering.

[0093] In one embodiment, on the basis of the above embodiment, the k value in the adjusted Wiener filtering function w2(u, v) in step S20 can be dynamically adjusted according to the local characteristics of the image. The image characteristics in different regions (such as texture complexity, noise level, signal strength, etc.) may be different. Therefore, using different k values in different regions can more precisely control the filtering method.

[0094] Optionally, the image to be processed is divided into multiple small blocks, and for each small block, its characteristic parameters are calculated, such as variance (reflecting texture complexity), noise estimation value, etc. A mapping relationship between the k value and the characteristic parameters is established according to the characteristic parameters. For example, a regression model (storing the mapping relationship between the k value and the characteristic parameters) can be pre-trained, with the characteristic parameters as the input and the k value as the output. During the filtering process, the k value is adjusted in real time according to the characteristic parameters of each small block.

[0095] This adaptive k-value adjustment method can dynamically adjust the k value according to the local characteristics of the image, so as to achieve better filtering effects in different regions.

[0096] In one embodiment, based on the above embodiment, the step of automatically settling the highway toll according to the payment method pre-signed by the vehicle associated with the license plate number in the vehicle-related payment system includes:

[0097] If the payment method pre-signed by the vehicle associated with the license plate number in the vehicle-related payment system is online payment, the corresponding online payment method is used to automatically settle the highway toll.

[0098] In this embodiment, the vehicle-related payment system queries the payment method pre-signed by the vehicle in the database according to the recognized license plate number. If the query result shows online payment, the system will further obtain the detailed information of the signed account.

[0099] For different online payment types, the information extracted by the system is different:

[0100] (1) Digital RMB account: Extract information such as the identifier of the Digital RMB wallet and the affiliated operating institution.

[0101] (2) Third-party payment platform account (such as WeChat, Alipay): Obtain information such as the bound third-party payment account and the platform identifier.

[0102] (3) Bank card binding account: Read information such as the bank card number (some may be desensitized), the issuing bank, and the card type (debit card or credit card).

[0103] Optionally, for Digital RMB payment, the vehicle-related payment system establishes a connection with the payment interface of the Digital RMB operating institution, and encapsulates necessary data such as the payment amount, order number, license plate number, and Digital RMB wallet information into a payment request and sends it to the operating institution. After receiving the payment request, the Digital RMB operating institution will verify the legality of the request, the wallet balance, etc. If the verification passes, the operating institution will deduct the corresponding amount from the vehicle owner's Digital RMB wallet to the highway toll account.

[0104] Optionally, for payments made through a third-party payment platform, the system calls the payment interface of the third-party payment platform and transmits information such as the payment amount, order number, license plate number, and the bound third-party payment account to the payment platform. The third-party payment platform verifies the accuracy of the payment information and the available balance or credit limit of the account. If the payment conditions are met, the platform will deduct the corresponding fees from the user's account and settle the funds to the designated account for highway tolls.

[0105] Optionally, for bank card payments, the vehicle-related payment system communicates with the bank system through a payment gateway and sends a payment instruction containing the bank card number, payment amount, order number, etc. The bank system conducts various verifications on the bank card, such as validity, balance, password (required for some transactions), etc. After successful verification, the bank deducts the corresponding highway toll from the bank card account and clears the funds to the designated toll collection account.

[0106] Regardless of the online payment method, after the payment is completed, the payment institution (digital currency operation institution, third-party payment platform, or bank) will feedback the payment result (success or failure) to the vehicle-related payment system.

[0107] If the payment is successful, the vehicle-related payment system will record the detailed information of the successful payment, such as payment time, payment amount, payment method, transaction serial number, etc. At the same time, update the vehicle's passing record, mark that this fee has been settled, and control the barrier to lift to allow the vehicle to pass.

[0108] If the payment fails, the vehicle-related payment system will immediately mark this transaction as a failed status and record the reason for failure. At the same time, prohibit the vehicle from passing and output corresponding prompt information according to the reason for failure, such as "Insufficient balance, please recharge or change the payment method in time", "Payment system failure, please wait or contact the staff", etc.

[0109] In this way, the vehicle owner does not need to stop at the toll station to make change or conduct cumbersome cash transactions. The vehicle can pass through the toll station quickly, greatly saving the passing time, improving the vehicle passing efficiency at the highway intersection, and reducing traffic congestion. At the same time, the online payment operation is fully automated without manual intervention. The vehicle owner only needs to sign up in advance, and the subsequent passing fees will be automatically deducted, which greatly facilitates travel.

[0110] In one embodiment, based on the above embodiment, the step of automatically settling the highway toll according to the payment method pre-signed by the vehicle associated with the license plate number in the vehicle-related payment system includes:

[0111] If the payment method pre-signed by the vehicle associated with the license plate number in the vehicle-related payment system is prepaid, deduct the corresponding highway toll from the prepaid amount;

[0112] If the prepaid amount is fully deducted, the expressway toll settlement is deemed successful; and if there is any excess in the prepaid amount, the excess amount will be refunded to the original route;

[0113] If the prepaid amount cannot be fully deducted, a prompt message will be output for the car owner to manually pay the remaining amount.

[0114] In this embodiment, the vehicle payment system uses the recognized license plate number to search the database for the vehicle's prepaid account information and obtain the current prepaid amount. The system compares the calculated highway toll with the prepaid amount and attempts to deduct the corresponding fee from the prepaid amount.

[0115] If the prepaid amount is sufficient to cover the toll, the vehicle payment system will determine that the toll settlement is successful. If there is any remaining amount after deducting the fees, the system will refund the remaining amount through the pre-agreed payment channel. For example, if the vehicle owner prepaid with a bank card, the remaining amount will be refunded to the bank card; if the prepaid amount was made through a third-party payment platform, the remaining amount will be refunded to the corresponding third-party payment account. The system will also record the refund amount, time, and transaction serial number.

[0116] If the prepaid amount is insufficient to cover the toll, the vehicle payment system will immediately display a prompt message. This prompt message can be displayed in eye-catching text on the display screen at the highway exit, and the driver can be informed through a voice broadcast device. The content is roughly as follows: "Your prepaid amount is insufficient. Please manually pay the remaining toll of X yuan."

[0117] At this time, the car owner needs to manually select other payment methods according to the prompts to complete the payment of the remaining amount. After the payment is successful, the system will update the pass record and release the vehicle.

[0118] In this way, car owners only need to prepay the fees in advance, and the fees will be automatically deducted when passing. There is no need to stop to pay, which saves time, improves traffic efficiency at highway intersections, and reduces congestion.

[0119] In one embodiment, based on the above embodiment, if the vehicle associated with the license plate number has pre-signed a payment method of prepaid in the vehicle-related payment system, before the step of deducting the corresponding highway toll from the prepaid amount, the step further includes:

[0120] When the vehicle-related payment system receives the scheduled toll information sent by the associated device of the vehicle whose payment method is prepaid, it calls the billing model to calculate the corresponding estimated toll fee;

[0121] Based on the estimated toll, prepayment information is sent to the associated device of the corresponding vehicle to prompt the owner to pay the corresponding amount in advance.

[0122] In this embodiment, when a prepaid vehicle's associated device (such as the owner's mobile app or in-vehicle smart terminal) sends scheduled travel information to the vehicle-related payment system, the system will promptly capture this information. This scheduled travel information may include key information such as the vehicle's planned starting point, destination, estimated travel time, and vehicle type.

[0123] The vehicle-related payment system calls the billing model provided by the highway management system based on the received scheduled passage information, and combines various parameters in the scheduled passage information, such as mileage, charging standards corresponding to the vehicle model, to accurately calculate the estimated passage fee for the vehicle's scheduled passage.

[0124] The vehicle-related payment system generates prepayment information based on the calculated estimated tolls. This information may include important details such as the estimated toll amount, payment method instructions, and payment deadline. The system then sends the prepayment information to the vehicle's associated device, prompting the owner to prepay the corresponding amount. After receiving the prepayment information, the owner can complete the prepayment process according to the prompts, preparing for the subsequent automatic settlement of highway tolls.

[0125] This allows drivers to estimate toll fees and prepay them in advance, making travel planning clearer and avoiding the tedious process of using cash or other payment methods, saving time. For both the vehicle payment system and highway management, this helps lock in fees in advance, ensuring smooth toll collection and reducing toll disputes. Furthermore, prepayment helps optimize fund management, improves the efficiency of the entire highway toll collection system, and makes highway travel smoother and more convenient.

[0126] In one embodiment, based on the above embodiment, the predetermined passage information includes any one of the following:

[0127] The selected information of the highway entrance and exit sent by the associated device before the vehicle enters the highway;

[0128] The selected information of the highway exit sent by the associated device after the vehicle enters the highway, wherein the traffic information of the vehicle at the highway entrance is also used when calculating the estimated toll;

[0129] Notification information sent by the associated device upon a vehicle's arrival at a highway payment plaza. This notification information also includes the vehicle's highway entrance information and highway exit information associated with the highway payment plaza when calculating the estimated toll.

[0130] The associated device sends the navigation planning route before the vehicle enters the highway. When the vehicle-related payment system receives the navigation planning route, it extracts the corresponding highway passage trajectory and calculates the estimated toll.

[0131] In this embodiment, before the vehicle owner travels or enters the highway, the highway entrance and exit can be selected through an associated device (such as a mobile phone APP) before the vehicle enters the highway. At this time, after receiving this information, the vehicle-related payment system calls the corresponding charging model and quickly calculates the estimated toll in combination with the selected entrance and exit.

[0132] Alternatively, after the vehicle has entered the highway, the vehicle owner can select the highway exit through the associated device at a roadside service station, rest area, etc. At this time, in addition to obtaining the exit selection information, the system also calls the toll road information recorded when the vehicle entered the highway (such as the entrance time, entrance toll station number, etc.), and calculates the estimated toll by integrating the entrance and exit information and the toll standard.

[0133] Alternatively, when the vehicle arrives at the highway payment plaza, the vehicle owner can send a notification message through the associated device. The system not only obtains this notification, but also calls the toll road information of the vehicle when it entered the highway, as well as the highway exit information associated with the highway payment plaza, and accurately calculates the estimated toll based on this complete data.

[0134] Alternatively, before entering the highway, the vehicle owner sends a navigation planned route through the associated device. After receiving the route, the vehicle-related payment system extracts the highway toll road trajectory from it, calls the charging model, and calculates the estimated toll based on the information such as the highway sections and mileage involved in the trajectory in combination with the toll standard.

[0135] In one embodiment, there are multiple ways of estimating tolls with high flexibility. Whether the vehicle owner selects the entrance and exit before entering the highway, sends a navigation planned route, selects the exit after entering the highway, or sends a notification when arriving at the payment plaza, the vehicle owner can choose a suitable method according to the actual situation to fit different travel scenarios. The system can calculate the toll by combining multi-dimensional data such as entrance information, exit information, and toll road trajectory, making the estimated toll closer to the actual situation and avoiding overcharging or undercharging. Moreover, knowing the estimated toll in advance can make the vehicle owner have a clear idea, prepare for payment in advance, reduce the waiting time for toll road passage, improve the convenience and smoothness of highway passage, and optimize the overall travel experience.

[0136] In one embodiment, on the basis of the above embodiment, after the step that when it is detected that the recognized license plate number has an associated vehicle in the vehicle-related payment system, the vehicle-related payment system notifies the railing device at the highway entrance to lift the bar and release the vehicle and records the corresponding toll road information, it further includes:

[0137] The vehicle-related payment system receives the positioning signal generated by the vehicle's associated device on the highway based on the satellite positioning module and generates a vehicle trajectory;

[0138] Among them, when calculating the highway toll, the vehicle trajectory is used as one of the calculation factors.

[0139] In this embodiment, after the vehicle-related payment system detects that the recognized license plate number has an associated vehicle, notifies the railing device at the highway entrance to lift the bar for release and records the passing information, the vehicle-related payment system will also continuously receive positioning signals sent by the vehicle's associated devices (such as in-vehicle navigation, mobile phone APPs with positioning functions, etc.) with the help of the satellite positioning module. These positioning signals contain the geographical location information of the vehicle at different times on the highway. Based on the series of received positioning signals, the system accurately depicts the driving path of the vehicle on the highway, thereby generating a vehicle trajectory.

[0140] When calculating the highway toll, the vehicle trajectory becomes a key calculation factor. The toll standards for different highway sections may vary. Through the vehicle trajectory, the system can clearly understand which specific sections the vehicle has traveled and which specific areas it has passed through. Combining this detailed section information and the corresponding toll rules, the system can calculate the highway toll for this vehicle more accurately, avoiding toll errors caused by inaccurate estimation and ensuring the fairness and accuracy of toll collection. For example, certain sections may have higher tolls due to factors such as high construction costs and large traffic volumes. The vehicle trajectory can clarify whether the vehicle has passed through these special sections, and then calculate the toll reasonably.

[0141] Optionally, a number of edge computing nodes are reasonably deployed along the highway. These nodes can receive and process the satellite positioning signals sent by the vehicle's associated devices in real time. The edge computing nodes form an ad-hoc network through the Internet of Things technology. They can automatically adjust the communication link and data forwarding path according to the real-time data transmission requirements and network conditions, realizing efficient data sharing and collaborative processing. By preliminarily screening and processing the positioning data, the data transmission volume is reduced, and the system response speed is improved.

[0142] In one embodiment, based on the above embodiment, the artificial intelligence model encodes the estimated output of the motion blur estimation branch to generate a weight matrix, and performs weighted fusion of the weight matrix with the first feature map of the spectral channel estimation branch to obtain an adjusted first feature map.

[0143] In this embodiment, the motion blur estimation branch receives the original vehicle image as input, analyzes and estimates the motion blur situation of the image through a series of neural network layers such as convolutional layers and pooling layers. Finally, it outputs a parameter vector p related to motion blur. This vector contains information about the degree and direction of the motion blur of the image, that is, p = [p1, p2, ⋯, p m , where m is the dimension of the vector, and each element represents different aspects of motion blur information.

[0144] In order to apply the motion blur parameters to the feature map adjustment of the spectral channel estimation branch, it is necessary to encode the estimated output to generate a weight matrix. Let the encoding function be g, which maps the motion blur parameter vector p to a weight matrix Z. The size and shape of this weight matrix match the first feature map of the spectral channel estimation branch for subsequent weighted fusion operations.

[0145] For example, if the size of the first feature map of the spectral channel estimation branch is D×L×C (where D is the height, L is the width, and C is the number of channels), then the size of the weight matrix Z can be C×C, and the encoding process can be expressed as: Z = g(p).

[0146] The spectral channel estimation branch also receives the original vehicle image as input. After a series of operations such as convolution and pooling, the first feature map is obtained, which contains the feature information of the image in different spectral channels.

[0147] Then, the generated weight matrix Z is weighted and fused with the first feature map of the spectral channel estimation branch. That is, for each position (i, j) in the first feature map, the feature vector in the channel dimension is multiplied by the weight matrix Z through matrix multiplication to obtain the adjusted feature vector f i,j ' = Z × f i,j .

[0148] Combining the adjusted feature vectors f i,j ' at all positions, the adjusted first feature map F1 is obtained. This adjusted first feature map F1 combines the motion blur parameters and spectral channel weight information, can more accurately reflect the features of the image, and provides a better basis for subsequent image deblurring and spectral optimization processing.

[0149] In one embodiment, based on the above embodiment, the artificial intelligence model performs a dot product operation on the estimated output of the spectral channel estimation branch and the second feature map of the motion blur estimation branch to obtain a tensor with the same shape as the second feature map;

[0150] The tensor is normalized to obtain the attention weights;

[0151] The attention weights are multiplied element-wise with the second feature map in the spatial dimension to obtain the adjusted second feature map.

[0152] In this embodiment, in the artificial intelligence model based on the dual-branch network, the motion blur estimation branch is responsible for estimating the motion blur parameters of the image, and the spectral channel estimation branch is responsible for estimating the weights of different spectral channels of the image.

[0153] The spectral channel estimation branch receives the original vehicle image. After a series of neural network operations such as convolution and pooling, it outputs the estimation results of the weights of different spectral channels in the image. For example, if the estimated output is a tensor S with a shape of C (number of channels), it reflects the importance weights of each spectral channel.

[0154] During the processing of the original image by the motion blur estimation branch, a series of feature maps are obtained as the second feature map.

[0155] Then, the dot product operation is performed between the estimated output S of the spectral channel estimation branch and the second feature map of the motion blur estimation branch. The purpose is to incorporate the weight information of the spectral channels into the feature map of the motion blur estimation branch, enabling the feature map to better reflect the importance of different spectral channels.

[0156] For the channel vector at each spatial position (i,j) in the second feature map , the dot product operation is performed with the estimated output of the spectral channel estimation branch. That is, for each channel K (1 ≤ K ≤ C), calculate . Combining the calculation results at all positions, a tensor T with the same shape as the second feature map is obtained.

[0157] The obtained tensor T is normalized to map its element values to the interval [0,1], so that it can be used as attention weights.

[0158] The Softmax function is used to normalize the attention weights. That is, for the channel vector at each spatial position (i,j) in the tensor T , calculate the attention weight vector , where:

[0159] ;

[0160] Then, all the attention weight vectors a i,j at all positions are combined to obtain the attention weight tensor A. Then, the attention weight tensor A and the second feature map of the motion blur estimation branch are multiplied element by element in the spatial dimension to enhance the expression of important features and suppress the influence of unimportant features.

[0161] For the channel vector at each spatial position (i,j) in the second feature map and the and in the attention weight tensor A, multiply them element by element to obtain the adjusted channel vector , where . Combining all the adjusted channel vectors q i,j at all positions, the adjusted second feature map F2 is obtained.

[0162] In this way, the artificial intelligence model realizes the adjustment of the feature map of the motion blur estimation branch by the spectral channel estimation branch, enhances the expression of important features in the feature map by using the attention mechanism, improves the model's ability to capture and process image features, and helps to more accurately recognize license plate numbers.

[0163] In addition, an in-vehicle payment system is provided in an embodiment of the present application. The internal architecture of the in-vehicle payment system can be as Figure 3 shown, including a processor, a memory, a communication interface, and an input interface connected by a system bus. Among them, the processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database is used to store data called by the computer program. The communication interface is used to communicate with an external terminal. The input interface is used to receive signals input by an external device. When the computer program is executed by the processor, it implements an in-vehicle payment method for passing through a highway as described in the above embodiments.

[0164] Those skilled in the art can understand that Figure 3 the structure shown in

[0165] is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the in-vehicle payment system to which the solution of the present application is applied. For example, in some alternative embodiments, the in-vehicle payment system may further include an output interface (not shown in the figure), and the output interface is also connected to the system bus and used to output corresponding signals to external devices.

[0166] In summary, the vehicle-related payment method, vehicle-related payment system, and computer-readable storage medium provided in the embodiments of the present application adopt a special dual-branch network pre-trained artificial intelligence model. Even if the vehicle is moving at high speed, resulting in blurred images, or the lighting conditions on the highway are complex, it can generate clear and accurate fused images for license plate recognition, meeting the accurate recognition requirements in complex highway scenarios. Moreover, once the recognition is successful, the vehicle-related payment system will respond quickly and automatically settle the fees and release the vehicle according to the payment method pre-signed by the vehicle, without manual intervention, reducing the vehicle waiting time and providing the vehicle owner with an efficient and convenient high-speed passage payment experience. Furthermore, throughout the vehicle-related payment and passage process, the vehicle owner does not need to pre-install devices such as ETC for high-speed passage billing, reducing the cost of vehicle passage on the highway.

[0167] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium provided in the present application and used in the embodiments can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0168] It should be noted that in this text, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such a process, device, article or method. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, device, article or method including such an element.

[0169] The above are only the preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, is equally included in the patent protection scope of the present application.

Claims

1. A vehicle-related payment method for highway passage, characterized in that, Including: Collecting vehicle images of vehicles to pass through based on cameras deployed at highway entrances; Using a pre-trained artificial intelligence model to identify the license plate numbers corresponding to the vehicle images; wherein, the artificial intelligence model adopts a dual-branch network, the motion blur estimation branch is used to estimate the motion blur parameters of the image, the spectral channel estimation branch is used to estimate the weights of different spectral channels of the image, and an interaction mechanism is introduced between the outputs of the two branches, so that the estimated outputs of each branch act on the feature map adjustment of other branches; and, in each branch, respectively using the estimated output of each branch and the adjusted feature map to process the original image, and fusing the images processed by each branch to generate a fused image after deblurring and spectral optimization for license plate number recognition; When it is detected that the identified license plate number has an associated vehicle in the vehicle-related payment system, the vehicle-related payment system notifies the railing device at the highway entrance to lift the bar and release the vehicle and records the corresponding passing information; When the camera deployed at the highway exit collects the vehicle image of the vehicle to pass through and uses the artificial intelligence model to identify the corresponding license plate number, the vehicle-related payment system calls the billing model provided by the highway management system, and calculates the highway toll according to the passing information of the current identified license plate number at the highway entrance and the passing information at the current highway exit; wherein, if the camera deployed at the highway section toll collection point collects the corresponding vehicle image and uses the artificial intelligence model to identify the corresponding license plate number, the passing information of the highway section toll collection point associated with the license plate number is used as one of the calculation factors for the highway toll; Automatically settling the highway toll according to the payment method pre-signed by the vehicle associated with the license plate number in the vehicle-related payment system; After the highway toll settlement is successful, the vehicle-related payment system notifies the railing device at the highway exit to lift the bar and release the vehicle; Wherein, the artificial intelligence model encodes the estimated output of the motion blur estimation branch to generate a weight matrix, and performs weighted fusion of the weight matrix and the first feature map of the spectral channel estimation branch to obtain an adjusted first feature map; using the estimated output of the spectral channel estimation branch and the first feature map to process the original image to obtain a spectral optimization map after removing the influence of motion blur; The artificial intelligence model performs a dot product operation on the estimated output of the spectral channel estimation branch and the second feature map of the motion blur estimation branch to obtain a tensor with the same shape as the second feature map; performing normalization processing on the tensor to obtain an attention weight; multiplying the attention weight and the second feature map element by element in the spatial dimension to obtain an adjusted second feature map; and deblurring the original image according to the adjusted second feature map in combination with the previously estimated motion blur parameters to obtain a clear image after deblurring; Based on the clear image after deblurring processed by the motion blur estimation branch and the spectral optimization map processed by the spectral channel estimation branch, fusing to generate a fused image after deblurring and spectral optimization.

2. The vehicle-related payment method for highway passage according to claim 1, characterized in that, The step of automatically settling the highway toll according to the payment method pre-signed by the vehicle associated with the license plate number in the vehicle-related payment system includes: If the vehicle associated with the license plate number has pre-signed an online payment method in the vehicle-related payment system, the corresponding online payment method will be used to automatically settle the highway toll.

3. The vehicle-related payment method for highway passage according to claim 1, characterized in that, The steps of automatically settling highway tolls based on the payment method pre-signed by the vehicle associated with the license plate number in the vehicle-related payment system include: If the vehicle associated with the license plate number has pre-signed a payment method of prepaid in the vehicle-related payment system, the corresponding highway toll will be deducted from the prepaid amount; If the prepaid amount is fully deducted, the expressway toll settlement is deemed successful; and if there is any excess in the prepaid amount, the excess amount will be refunded to the original route; If the prepaid amount cannot be fully deducted, a prompt message will be output for the car owner to manually pay the remaining amount.

4. The vehicle-related payment method for highway passage according to claim 3, characterized in that, If the vehicle associated with the license plate number has pre-signed a payment method of prepaid in the vehicle-related payment system, before deducting the corresponding highway toll from the prepaid amount, the method further includes: When the vehicle-related payment system receives the scheduled toll information sent by the associated device of the vehicle whose payment method is prepaid, it calls the billing model to calculate the corresponding estimated toll fee; Based on the estimated toll, prepayment information is sent to the associated device of the corresponding vehicle to prompt the owner to pay the corresponding amount in advance.

5. The vehicle-related payment method for highway passage according to claim 4, wherein The predetermined passage information includes any one of the following: The selected information of the highway entrance and exit sent by the associated device before the vehicle enters the highway; The selected information of the highway exit sent by the associated device after the vehicle enters the highway, wherein the traffic information of the vehicle at the highway entrance is also used when calculating the estimated toll; Notification information sent by the associated device upon a vehicle's arrival at a highway payment plaza. This notification information also includes the vehicle's highway entrance information and highway exit information associated with the highway payment plaza when calculating the estimated toll. The associated device sends the navigation planning route before the vehicle enters the highway. When the vehicle-related payment system receives the navigation planning route, it extracts the corresponding highway passage trajectory and calculates the estimated toll.

6. The vehicle-related payment method for highway passage according to claim 1, characterized in that After the step of detecting that the identified license plate number exists in the vehicle-related payment system and the vehicle-related payment system notifies the barrier device at the highway entrance to lift the barrier and release the vehicle and record the corresponding passage information, the method further includes: The vehicle-related payment system is based on a satellite positioning module, which receives positioning signals generated by the vehicle's associated equipment on the highway and generates vehicle tracks; Among them, when calculating highway tolls, vehicle trajectory is used as one of the calculation factors.

7. A vehicle-related payment system, characterized in that, The vehicle-related payment system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the vehicle-related payment method for highway travel as described in any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the vehicle-related payment method for highway travel as described in any one of claims 1 to 6.

Images