Vehicle fee payment system and method

Abstract

The embodiment of the present application discloses a vehicle fee payment system and method, which includes: an information collection device, installed on the main driver's side of the vehicle, connected to a control unit, for collecting information on vehicle fees to be paid, and transmitting the information on vehicle fees to be paid to the control unit; a control unit, located in the vehicle, connected to a vehicle network communication terminal, for transmitting the information on vehicle fees to be paid to the vehicle network communication terminal; a vehicle network communication terminal, located in the vehicle, wirelessly communicating with a mobile terminal, for sending the information on vehicle fees to be paid to the mobile terminal; a mobile terminal, for executing a vehicle fee payment operation based on the information on vehicle fees to be paid. The embodiment of the present application realizes remote payment of vehicle fees in the case of separation of the person and the vehicle, solves the problem of remote payment in smart summoning, and can avoid vehicle congestion caused by failure to pay vehicle fees in a timely manner.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of Internet of Things technology, and in particular to a vehicle fee payment system and method. Background Art

[0002] ADAS (Advanced Driving Assistance System) of new energy vehicles uses various sensors installed on the vehicle (such as smart cameras, millimeter-wave radars, lidars, GNSS, etc.) to sense the surrounding environment at any time during the vehicle's driving, collect data, identify, detect and track dynamic and static objects, and combine high-precision map data to perform systematic calculations and planning controls to effectively avoid foreseeable dangers while increasing vehicle comfort. In addition, the development of additional humanized functions in combination with related sensors can make new energy vehicles more intelligent.

[0003] HPP (Home Zone Parking Pilot), an increasingly popular driver assistance system, enhances convenience and travel efficiency in everyday life, gaining widespread recognition and attention. Intelligent Summon Mode, an extension of HPP, is a cutting-edge technology currently available in new energy vehicles. It utilizes a domain controller, high-definition cameras, ultrasonic radar, an inertial navigation system comprised of GNSS, GPS, RTK, and IMU, SD maps, the cloud, and a mobile app to enable remote summoning of vehicles.

[0004] However, when users use the smart summoning function, the vehicle summoned from the parking space to the designated location (for example, from the underground parking space to the designated location on the ground) generally needs to pass through the toll gate to process the payment task. If it is in a shopping mall area with temporary parking, the existing technology cannot pay the vehicle fee when people and vehicles are separated. The smart summoning function will not be able to pass through the toll gate smoothly, resulting in the point-to-point function of the summoning function not being fully realized, and causing subsequent vehicle congestion. Summary of the Invention

[0005] The embodiments of the present application provide a vehicle fee payment system and method to enable remote payment of vehicle fees when a person and a vehicle are separated, thereby avoiding vehicle congestion.

[0006] In order to solve the above problems, in a first aspect, an embodiment of the present application provides a vehicle fee payment system, comprising:

[0007] An information collection device, mounted on the driver's side of the vehicle and connected to the control unit, is used to collect information about vehicle fees to be paid and transmit the information to the control unit;

[0008] a control unit, located in the vehicle and connected to the vehicle networking communication terminal, for transmitting the vehicle fee information to be paid to the vehicle networking communication terminal;

[0009] a vehicle networking communication terminal, located in the vehicle, in wireless communication with the mobile terminal, and configured to send the vehicle fee information to be paid to the mobile terminal;

[0010] The mobile terminal is used to perform a vehicle fee payment operation according to the vehicle fee information to be paid.

[0011] In a second aspect, an embodiment of the present application provides a vehicle fee payment method, comprising:

[0012] collecting vehicle fee information to be paid through an information collection device and transmitting the vehicle fee information to be paid to a control unit;

[0013] transmitting the to-be-paid vehicle fee information to the Internet of Vehicles communication terminal via the control unit;

[0014] The vehicle fee information to be paid is sent to the mobile terminal through the Internet of Vehicles communication terminal, and the mobile terminal is used to perform a vehicle fee payment operation according to the vehicle fee information to be paid.

[0015] The vehicle fee payment system and method provided in the embodiments of the present application collect vehicle fee information to be paid through an information collection device installed on the driver's side of the vehicle, and transmit the vehicle fee information to be paid to a control unit. The control unit sends the vehicle fee information to be paid to a mobile terminal through a vehicle network communication terminal. The mobile terminal can perform vehicle fee payment operations based on the vehicle fee information to be paid, realizing remote payment of vehicle fees when the person and the vehicle are separated, solving the problem of remote payment in smart summoning, and avoiding vehicle congestion caused by failure to pay vehicle fees in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] Figure 1 This is a schematic diagram of the structure of a vehicle fee payment system provided in an embodiment of the present application;

[0018] Figure 2 This is a schematic diagram of the structure of the advanced driver assistance system in an embodiment of the present application;

[0019] Figure 3 This is a flow chart of a vehicle fee payment method provided in an embodiment of the present application;

[0020] Figure 4 Schematic diagram of the comparison between the priori frame and the real bounding box in the embodiment of the present application;

[0021] Figure 5 This is a flow chart of a vehicle fee payment method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0023] Figure 1 This is a schematic diagram of the structure of a vehicle fee payment system provided by an embodiment of the present application. Figure 1 As shown, the vehicle fee payment system includes: an information collection device 10 , a control unit 20 , a vehicle network communication terminal 30 and a mobile terminal 40 .

[0024] The information collection device 10 is installed on the driver's side of the vehicle and is connected to the control unit for collecting information on vehicle fees to be paid and transmitting the information to be paid to the control unit.

[0025] The control unit 20 is located in the vehicle and is connected to the vehicle networking communication terminal, and is used to transmit the vehicle fee information to be paid to the vehicle networking communication terminal;

[0026] The vehicle networking communication terminal 30 is located in the vehicle and wirelessly communicates with the mobile terminal to send the vehicle fee information to be paid to the mobile terminal;

[0027] The mobile terminal 40 is used to perform a vehicle fee payment operation according to the vehicle fee information to be paid.

[0028] The information collection device 10 is mounted on the driver's side of the vehicle and can collect information about tolls to be paid at the toll gate. For example, the information collection device 10 can be an image collection device for capturing images of tolls to be paid at the toll gate; alternatively, the information collection device 10 can be a sensor or other device for collecting tolls to be paid at the toll gate; alternatively, the information collection device 10 can be a near-field communication device that can communicate with a device carrying toll information at the toll gate to collect the toll information.

[0029] The control unit 20 can be a domain control module (ADAS Domain Controller System, ADCS) in the advanced driving assistance system in the vehicle, which is used to control the remote payment of vehicle fees. It can establish communication with the mobile terminal used by the user to send the vehicle fee information to be paid collected by the information collection device 10 to the mobile terminal 40 through the Internet of Vehicles communication terminal 30, so that the mobile terminal 40 can perform the vehicle fee payment operation based on the vehicle fee information to be paid to complete the remote payment of vehicle fees when the person and the vehicle are separated.

[0030] The IoV communication terminal 30 is an onboard T-BOX (Telematic Box) used to establish communication between the mobile terminal and the control unit 20 in the advanced driver assistance system. In the embodiment of the present application, the IoV communication terminal 30 can send information about vehicle fees to be paid to the mobile terminal, and can receive feedback from the mobile terminal regarding the information about vehicle fees to be paid, and send the feedback information to the control unit 20. The communication between the IoV communication terminal 30 and the mobile terminal 40 is wireless communication, for example, using 4G signal communication or 5G signal communication.

[0031] The mobile terminal 40 is a terminal device used by the user, such as a mobile phone, tablet computer, etc. It can display the vehicle fee information to be paid sent by the vehicle network communication terminal 30, and allow the user to operate to execute the vehicle fee payment operation, or it can also reject the vehicle fee payment operation.

[0032] The vehicle fee payment system provided in this embodiment collects the vehicle fee information to be paid through an information collection device installed on the driver's side of the vehicle, and transmits the vehicle fee information to be paid to a control unit. The control unit sends the vehicle fee information to be paid to the mobile terminal through the Internet of Vehicles communication terminal. The mobile terminal can perform vehicle fee payment operations based on the vehicle fee information to be paid, realizing remote payment of vehicle fees when the person and the vehicle are separated, solving the problem of remote payment in smart summoning, and avoiding traffic congestion caused by failure to pay vehicle fees in time.

[0033] On the basis of the above technical solution, the information acquisition device is an image acquisition device, which is used to: acquire an image including the information of the vehicle fee to be paid, and transmit the image to the control unit.

[0034] The information acquisition device 10 may be an image acquisition device, which may be, for example, a camera. The image acquisition device is mounted on the driver's side and may capture an image of the vehicle fee information to be paid at the toll gate, and transmit the image to the control unit 20. The control unit 20 may transmit the image including the vehicle fee information to be paid to the vehicle networking communication terminal 30. The vehicle networking communication terminal 30 may send the image to the mobile terminal 40. The mobile terminal 40 may identify the vehicle fee information to be paid in the image and perform a vehicle fee payment operation based on the vehicle fee information to be paid. By using the image acquisition device as the information acquisition device, the vehicle fee information to be paid may be collected relatively quickly and conveniently, and this can be achieved without the need to install other equipment at the toll gate.

[0035] The image acquisition device is installed on the B-pillar of the vehicle, which can facilitate the acquisition of images including vehicle fee information to be paid at the payment gate, for example, it can facilitate the acquisition of images including a charging QR code.

[0036] On the basis of the above technical solution, in the image, the vehicle fee information is represented by a QR code, so that by collecting an image including the charging QR code, the vehicle fee information to be paid can be obtained.

[0037] Based on the above technical solution, the image acquisition device is further used to: perform convolution processing on the image, locate the QR code area in the image, extract the QR code area in the image based on the positioning result, obtain a QR code image, and transmit the QR code image to the control unit;

[0038] The control unit is specifically configured to: transmit the QR code image to the Internet of Vehicles terminal;

[0039] The vehicle network terminal is specifically used to: send the QR code image to the mobile terminal;

[0040] The mobile terminal is specifically used to: identify the two-dimensional code image, obtain vehicle fee information to be paid, and perform a fee payment operation according to the vehicle fee information to be paid.

[0041] After capturing an image, the image acquisition device processes the image to improve its clarity and extract the QR code therefrom. The image processing can be performed using a neural network-based image processing model, which can be a perception model based on a fully convolutional neural network. The image acquisition device performs convolution processing on the captured image using the image processing model to improve image clarity and detects the QR code therein. The device determines the center point coordinates, bounding box size, and corresponding confidence level of candidate regions in the image. Candidate regions with confidence levels greater than a confidence threshold are identified as QR code regions. Based on the center point coordinates and bounding box size of the QR code regions, the device extracts the QR code regions from the image to obtain a QR code image. After extracting the QR code image, the image acquisition device transmits the QR code image to a control unit, which transmits the QR code image to a connected vehicle terminal. The connected vehicle terminal then sends the QR code image to a mobile terminal, which recognizes the QR code image, obtains vehicle fare information to be paid, and performs a fare payment operation based on the vehicle fare information to be paid.

[0042] After the captured image is processed by the image acquisition device and the QR code image is extracted, it only needs to be sent to the mobile terminal, which reduces the amount of transmitted data and improves the clarity of the QR code image, making it easier for the mobile terminal to recognize the QR code and perform payment operations.

[0043] It should be noted that the above-mentioned operation of processing the image to extract the QR code image can also be performed by the control unit, that is, after the image acquisition device acquires the image, the image can be directly transmitted to the control unit, and the control unit uses the above-mentioned image processing model to process the image and extract the QR code image, and then sends the QR code to the mobile terminal through the Internet of Vehicles communication terminal, and the mobile terminal completes the vehicle fee payment operation.

[0044] Based on the above technical solution, the control unit is connected to the Internet of Vehicles communication terminal via a hard line, which can be, for example, a POS (Packet Over SONET / SDH) hard line, so that images can be quickly transmitted between the control unit and the Internet of Vehicles communication terminal.

[0045] Figure 2 is a schematic diagram of the structure of the advanced driver assistance system in the embodiment of the present application. Figure 2As shown, the advanced driver assistance system includes the above-mentioned vehicle fee payment system, that is, it includes an image acquisition device 11, a control unit 20, a vehicle network communication terminal 30, a mobile terminal 40, and also includes an intelligent central gateway 50, a body domain controller 60, a power domain controller 70, a steering wheel 80, as well as sensors 90, cameras 100 in other locations, etc. The control unit 20 and the intelligent central gateway 50 are connected via a CAN bus, the body domain controller 60, the power domain controller 70, and the steering wheel 80 are respectively connected to the intelligent central gateway 50, the sensors 90 and cameras 100 in other locations are connected to the control unit 20, and the vehicle network communication terminal 30 is also connected to the intelligent central gateway 50 via a CAN bus. Low Voltage Differential Signal (LVDS) can be transmitted between the camera 100 and the control unit 20.

[0046] The embodiments of the present application can be applied to scenarios where a vehicle is summoned from an underground garage to a designated location on the ground. If the vehicle passes through a toll gate, it can automatically pay (or user-assisted payment). In traditional application scenarios, for example, if a long-term resident has not entered the license plate information into the property parking management system, and for underground parking garages in large-scale event centers or commercial centers, when a vehicle is summoned to a designated location in a driver-vehicle separation scenario, if a toll gate is encountered, the image of the toll QR code collected when the vehicle arrives at the toll gate can be effectively transmitted to the mobile terminal via 5G signals, creating a timely and effective payment environment, facilitating users to make remote payments, solving the remote payment problem during smart summoning, and avoiding vehicle congestion. At the same time, the user's payment status and vehicle control status can be shared between the mobile terminal and the vehicle. That is, through the control unit, the user can understand the specific status of the vehicle in real time, and the vehicle can also convey the expected control of the vehicle based on the user's feedback information and then combined with its own perception of the environment, control logic, actuators, etc., which not only ensures the safe and autonomous driving of the vehicle, but also facilitates users to make remote payments.

[0047] Figure 3 This is a flowchart of a vehicle fee payment method provided in an embodiment of the present application. The vehicle fee payment method can be executed by the vehicle fee payment system described in the above embodiment, and includes the following steps.

[0048] Step 310: Collect the vehicle fee information to be paid through the information collection device, and transmit the vehicle fee information to be paid to the control unit.

[0049] The information collection device is mounted on the driver's side of the vehicle and can collect information about the vehicle tolls to be paid at the toll gate. For example, the information collection device can be an image acquisition device for capturing images containing the information about the vehicle tolls to be paid; or, the information collection device can be a sensor or other device for collecting the information about the vehicle tolls to be paid at the toll gate; or, the information collection device can be a near-field communication device that can communicate with a device carrying the information about the vehicle tolls to be paid at the toll gate to collect the information about the vehicle tolls to be paid.

[0050] After the information collection device collects the vehicle fee information to be paid at the payment gate, the vehicle fee information to be paid is transmitted to the control unit so that the control unit controls the vehicle network communication terminal to send the vehicle fee information to be paid to the mobile terminal.

[0051] Step 320: Transmitting the vehicle fee information to be paid to the Internet of Vehicles communication terminal via the control unit.

[0052] Step 330 : Sending the vehicle fee information to be paid to a mobile terminal via the Internet of Vehicles communication terminal, and the mobile terminal is used to perform a vehicle fee payment operation according to the vehicle fee information to be paid.

[0053] The IoV communication terminal, i.e., the vehicle-mounted T-BOX, is used to establish communication between the mobile terminal and the control unit in the advanced driver assistance system. In the embodiment of the present application, the IoV communication terminal can transmit information about vehicle fees to be paid to the mobile terminal, and can receive feedback from the mobile terminal regarding the information about vehicle fees to be paid, and transmit the feedback information to the control unit. The communication between the IoV communication terminal and the mobile terminal is wireless communication, for example, using 4G or 5G signal communication.

[0054] The mobile terminal is a terminal device used by the user, such as a mobile phone, tablet computer, etc. It can display the vehicle fee information to be paid sent by the vehicle network communication terminal and allow the user to operate to execute the vehicle fee payment operation, or it can also reject the vehicle fee payment operation.

[0055] The vehicle fee payment method provided in the embodiment of the present application collects vehicle fee information to be paid through an information collection device, and transmits the vehicle fee information to be paid to a control unit, which transmits the vehicle fee information to be paid to a vehicle network communication terminal through the control unit, and sends the vehicle fee information to be paid to a mobile terminal through the vehicle network communication terminal. The mobile terminal executes the vehicle fee payment operation according to the vehicle fee information to be paid, thereby realizing remote payment of vehicle fees when the person and the vehicle are separated, solving the problem of remote payment in smart summoning, and avoiding vehicle congestion caused by failure to pay vehicle fees in time.

[0056] On the basis of the above technical solution, the information acquisition device is an image acquisition device;

[0057] Collecting the vehicle fee information to be paid through the information collection device and transmitting the vehicle fee information to be paid to the control unit includes: collecting an image including the vehicle fee information to be paid through the image collection device and transmitting the image to the control unit.

[0058] The information acquisition device may be an image acquisition device, which may be, for example, a camera. The image acquisition device is mounted on the driver's side of the vehicle, preferably on the B-pillar of the vehicle, and may capture an image of the vehicle fee information to be paid at the payment gate, and transmit the image to a control unit. The control unit may transmit the image including the vehicle fee information to be paid to a vehicle network communication terminal, which may transmit the image to a mobile terminal. The mobile terminal may recognize the vehicle fee information to be paid in the image and perform a vehicle fee payment operation based on the vehicle fee information to be paid. By using the image acquisition device as the information acquisition device, the vehicle fee information to be paid may be collected relatively quickly and conveniently, and this can be achieved without the need to install other equipment at the payment gate.

[0059] Based on the above technical solution, the image acquisition device acquires an image including the vehicle fee information to be paid and transmits the image to the control unit, including: acquiring an image including the vehicle fee information to be paid by the image acquisition device; performing convolution processing on the image using an image processing model by the image acquisition device, and locating the QR code area in the image, extracting the QR code area in the image based on the positioning result to obtain a QR code image, and transmitting the QR code image to the control unit;

[0060] The vehicle fee information to be paid is transmitted to the Internet of Vehicles communication terminal through the control unit, including: transmitting the QR code image to the Internet of Vehicles communication terminal through the control unit; sending the vehicle fee information to be paid to the mobile terminal through the Internet of Vehicles communication terminal, including: sending the QR code image to the mobile terminal through the Internet of Vehicles communication terminal, the mobile terminal is used to identify the QR code image, obtain the vehicle fee information to be paid, and perform the fee payment operation according to the vehicle fee information to be paid.

[0061] After capturing an image, the image acquisition device processes the image to improve its clarity and extract the QR code therefrom. A neural network-based image processing model can be used to process the image. The image acquisition device performs convolution processing on the captured image using the image processing model to improve its clarity and detects the QR code within the image. The device determines the center point coordinates, bounding box size, and corresponding confidence level of a candidate region within the image. Candidate regions with confidence levels greater than a confidence threshold are identified as QR code regions. Based on the center point coordinates and bounding box size of the QR code region, the QR code region is extracted from the image to obtain a QR code image. After extracting the QR code image, the image acquisition device transmits the QR code image to a control unit, which transmits the QR code image to a connected vehicle terminal. The connected vehicle terminal then sends the QR code image to a mobile terminal, which can recognize the QR code image, obtain vehicle fee information to be paid, and perform a fee payment operation based on the vehicle fee information to be paid.

[0062] After the captured image is processed by the image acquisition device and the QR code image is extracted, it only needs to be sent to the mobile terminal, which reduces the amount of transmitted data and improves the clarity of the QR code image, making it easier for the mobile terminal to recognize the QR code and perform payment operations.

[0063] The image processing model is a perception model based on a fully convolutional neural network and has been pre-trained. The training of the image processing model uses a rolling time domain optimization algorithm to solve the control variables for multiple targets. The steps include: ① determining the center coordinates (x, y) of the QR code area; ② determining the bounding box size (w, h) of the QR code area; ③ setting the confidence level (Conf); and ④ minimizing the deviation between the predicted grid and the true target bounding box. Confidence is expressed as:

[0064] Conf=Pr(Object)×IOU(R pred , R GT )

[0065]

[0066] Among them, Pr(Object) represents the probability that the predicted bounding box contains a QR code, IOU(R pred , R GT ) represents the overlap ratio between the real bounding box and the predicted bounding box, where R pred Represents the predicted bounding box, R GT The IOU overlap rate (or Jaccard overlap rate) is the ratio of the intersection and union of the real bounding box and the predicted bounding box.

[0067] As mentioned above, each predicted bounding box requires a total of 5 dimensions of features: the center coordinates (x, y) of the predicted bounding box, the size of the bounding box (w, h), and the confidence level. During training, the sample image can be divided into a preset number of grids. When the center of no target (charge QR code) falls into a certain grid, or the center of a target falls into the grid, but the overlap rate between some prior boxes in the grid and the target is not the maximum, the network will still use these prior boxes as the benchmark for prediction output. For the S×S grid of the prediction port of the convolutional neural network, a feature map of size S×S×B×5 (5 dimensional features) will be output, where B represents the number of prior boxes contained in each grid. Figure 4 : is a schematic diagram comparing the a priori frame and the real bounding box in the embodiment of the present application, such as Figure 4 As shown in the figure, the centers of the priori box 1 and the true bounding box 2 are located at the center of the same grid. The network will predict the output based on the priori box. The goal of training the image processing model is to bring the priori box close to the true box. The loss function used is as follows:

[0068]

[0069]

[0070]

[0071] Among them, Loss loc Represents the loss function of the image processing model, which consists of two parts. Represents the loss generated by the prior box in the grid responsible for predicting the target (charge QR code) position, represents the loss of the prior box in the grid that does not need to predict the target location, Indicates loss item The weight of Indicates loss item The weight, l ij Is an indicator function, indicating whether the content in the target box is the predicted target (charged QR code), the target box is the bounding box generated when the prior box approaches the real bounding box, b*={b x * , b y * , b w * , b h *} represents the position of the real bounding box on the output feature map, b'={b x ',b y ',b w ',b h '} represents the position of the predicted bounding box on the output feature map, p = {px ,p y ,p w ,p h} represents the position of the prior box on the output feature map.

[0072] When the center of the true bounding box b* falls within the center of a grid, that is, when the current grid is responsible for predicting the area of the toll QR code, the true value of the position offset of its predicted bounding box must satisfy the constraints of the Bernoulli distribution interval (also called the sigmoid function) and be between [0, 1].

[0073] The learning goal of the target detection positioning task (i.e. determining the QR code area) is to make the predicted bounding box b' close to the real bounding box position b*. The positioning task of the QR code area uses the deep training loss function Loss loc , using the least square error, where is the prior box p in the grid responsible for predicting the target location = {p x ,p y ,p w ,p h The loss generated is to minimize the deviation between the true bounding box position and the predicted bounding box position. It is the loss of the prior box in the grid that does not need to predict the target position, which is used to supervise the network model to make the predicted bounding box at this position close to the default prior box, that is, to keep the default prior box. Compared with the grid that needs to predict the bounding box (the center of the bounding box falls in the grid), a large number of grids do not need to predict the target. In order to make the loss term and The scale is similar, using weights and To balance the two losses.

[0074] The above-mentioned training process of the image processing model is as follows: obtaining a first sample image set, wherein the first sample image set includes multiple first sample images and the annotation of the real bounding box of the QR code area in each first sample image; inputting the first sample image into the image processing model, and the image processing model divides the first sample image into a preset number of grids, and predicts the bounding box based on each grid, and outputs the prediction result, wherein the prediction result includes the center coordinates of the predicted bounding box and the size of the predicted bounding box; determining the loss function value based on the prediction result and the annotation of the real bounding box corresponding to the first sample image; adjusting the network parameters of the image processing model based on the loss function value, and iteratively performing the training operation of the image processing model until the training end condition is met, the training ends, and a trained image processing model is obtained. The training end condition can be the convergence of the loss function value.

[0075] The image processing model trained through the above training process can improve the clarity of the image and accurately determine the QR code area where the charging QR code is located.

[0076] On the basis of the above technical solution, before the image acquisition device uses the image processing model to improve the clarity of the image, the method further includes: clustering the image using a clustering model by the image acquisition device;

[0077] The image acquisition device uses an image processing model to perform convolution processing on the image, including: if the clustering result indicates that the image is classified into a cluster including a QR code area, the image acquisition device uses an image processing model to perform convolution processing on the image.

[0078] The clustering model uses a K-means algorithm and is pre-trained based on a second sample image set. Some of the second sample images in the second sample image set include a charging QR code, while others do not. The clustering model aims to cluster second sample images that include a charging QR code into one category, and cluster other images of the same category into one category. The clustering model can output multiple clusters.

[0079] When training the clustering model, the second sample image set can be initialized into k clusters {τ i , i=1,…,k}, for each cluster, calculate the sample mean {m i , i=1,…,k} and the error sum of squares clustering criterion J e , where each cluster τ i The sample mean of is expressed as:

[0080]

[0081] Among them, N i is the i-th cluster τ i The number of samples in , y is the number of samples in cluster i τ i The second sample image in m i is the sample mean.

[0082] Error sum of squares clustering criterion J e Expressed as:

[0083]

[0084] Among them, J e Represents the error square sum clustering criterion, which is a function of the category set and the overall sample set. Through the iterative method, J e The smallest cluster is the optimal clustering result under the error square sum criterion.

[0085] From the i-th cluster τ i Take any sample y, that is, y∈τ i , if the i-th cluster τ i The number of the second sample image is not equal to 1, that is, N i ≠1, then calculate ρ j and ρ i :

[0086]

[0087]

[0088] Among them, m j represents the jth cluster τ j The sample mean of .

[0089] Determine ρ j The smallest one in ρ min , if ρ min <ρ i , then the sample y is removed from the cluster τ i Move to ρ min The corresponding cluster τ min and recalculate {m i , i = 1, ..., k} and J e , and iteratively perform the above from the i-th cluster τ i Take any sample y and calculate ρ j and ρ i , and based on ρ j and ρ i The clustering of sample y is adjusted until the iteration termination condition is met, that is, the error square sum clustering criterion J e When convergence is reached, the iteration is stopped and the trained clustering model is obtained. The trained clustering model can include a certain number of clusters. Assuming there are k clusters, the sample mean of each cluster {m i , i=1,…,k}.

[0090] The K-means algorithm has a good effect on the local feature search algorithm. For the local features of the toll QR code, when training the perceptron, it is only necessary to classify the typical QR code features. Once the local features converge, the usage conditions can be met. Therefore, the clustering model can be used to cluster the collected images first.

[0091] After the image is captured by the image capture device, the captured image is first clustered using a clustering model. If the clustering result indicates that the captured image belongs to a cluster that includes a QR code area, the image is further processed using an image processing model through the image capture device. If the clustering result indicates that the captured image belongs to a cluster that does not include a QR code area, the image is discarded.

[0092] When clustering the collected image using the clustering model, the image is classified into a cluster τ in the clustering model. i , and calculate the clustering τ i The corresponding ρi, and the corresponding ρj of other clusters, determine ρ j The smallest one in ρ min , if ρ min <ρ i , then move the image to ρ min The corresponding cluster τ min In the clustering result corresponding to the collected image, the clustering result τ min is the cluster to which the collected image belongs.

[0093] By clustering the collected images and determining whether to use the image processing model to process the images based on the clustering results, the amount of calculation can be reduced and the QR code image including the charging QR code can be quickly obtained.

[0094] On the basis of the above technical solution, before the information collection device collects the vehicle fee information to be paid, it also includes: when the control unit determines that there is a toll pole in front of the vehicle, a remote payment request is sent to the vehicle network communication terminal; the remote payment request is sent to the mobile terminal through the vehicle network communication terminal, and the instruction information returned by the mobile terminal is received, and the instruction information is transmitted to the control unit; when the control unit determines that the instruction information is a remote payment instruction, the information collection device is triggered to collect information.

[0095] During the process of intelligent summoning of a vehicle, if the vehicle stops, the control unit will determine whether there is a toll pole in front of the vehicle. The control unit can obtain the image captured by the front camera and determine whether there is a toll pole in front of the vehicle based on the image, or the control unit can obtain data from the front sensor and determine whether there is a toll pole in front of the vehicle based on the sensor data. If the control unit determines that there is a toll pole in front of the vehicle, it will send a remote payment request to the Internet of Vehicles communication terminal via the CAN bus, and the Internet of Vehicles communication terminal will send the remote payment request to the mobile terminal. When the mobile terminal receives the remote payment request, it can display the remote payment request based on the user's instructions. The user can give a remote payment instruction or reject the remote payment request. The mobile terminal sends the user's instruction information to the Internet of Vehicles communication terminal, and the Internet of Vehicles communication terminal transmits the instruction information to the control unit via the CAN bus. When the control unit determines that the instruction information is a remote payment instruction, it triggers the information collection device to collect information. When the control unit determines that the instruction information is a remote payment instruction to reject the remote payment, it exits the intelligent summoning function and requests to take over.

[0096] During the smart summoning process, if the vehicle passes through the payment gate, it can be determined whether to make remote payment based on the communication between the vehicle and the mobile terminal, so that the user can understand the vehicle status in a timely manner.

[0097] Figure 5 This is a flowchart of a vehicle fee payment method provided in an embodiment of the present application. The vehicle fee payment method can be executed by the vehicle fee payment system described in the above embodiment, and includes the following steps.

[0098] Step 510: When the control unit determines that there is a toll booth in front of the vehicle, a remote payment request is sent to the Internet of Vehicles communication terminal.

[0099] During smart summoning, when the vehicle stops, if the control unit determines that there is no toll booth in front of the vehicle, there is no need to enable the remote payment function.

[0100] Step 520: Send the remote payment request to the mobile terminal through the Internet of Vehicles communication terminal, receive the instruction information returned by the mobile terminal, and transmit the instruction information to the control unit.

[0101] Step 530: When the control unit determines that the instruction information is a remote payment instruction, the information collection device is triggered to collect information.

[0102] Step 540: Collect the vehicle fee information to be paid through the information collection device, and transmit the vehicle fee information to be paid to the control unit.

[0103] Step 550: Transmitting the vehicle fee information to be paid to the Internet of Vehicles communication terminal through the control unit.

[0104] Step 560: Send the vehicle fee information to be paid to the mobile terminal through the Internet of Vehicles communication terminal, and the mobile terminal is used to perform a vehicle fee payment operation according to the vehicle fee information to be paid.

[0105] Step 570: After receiving the payment success information of the mobile terminal through the Internet of Vehicles communication terminal, the payment success information is transmitted to the control unit.

[0106] Step 580: When the control unit determines that the toll collection bar in front of the vehicle is raised, the vehicle call is restored.

[0107] When the control unit determines that the toll collection pole in front of the vehicle has not been raised, it exits the summoning function and requests the mobile terminal to take over.

[0108] The embodiment of the present application can effectively transmit the image of the vehicle arriving at the toll gate, including the toll QR code, to the mobile terminal through a non-directional signal, creating a timely and effective payment environment, and at the same time sharing the user's payment status and vehicle control status. This means that through the control unit, the user can understand the specific status of the vehicle in real time, and the vehicle can also convey the expected control of the vehicle through the user's feedback information and then combine its own perception environment, control logic, actuators, etc., which not only ensures the safe and autonomous driving of the vehicle, but also facilitates remote payment for users, solving the problem of remote payment for remote summons.

[0109] Each embodiment in this specification is described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between the various embodiments can be referred to in conjunction with each other. For the device embodiments, since they are generally similar to the method embodiments, their description is relatively simple, and for relevant parts, reference can be made to the description of the method embodiments.

[0110] The above is a detailed introduction to a vehicle fee payment system and method provided in an embodiment of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0111] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

Claims

1. A vehicle fee payment system, characterized in that: include: An information collection device, mounted on the driver's side of the vehicle and connected to the control unit, is used to collect information about vehicle fees to be paid and transmit the information to the control unit; a control unit, located in the vehicle and connected to the vehicle networking communication terminal, for transmitting the vehicle fee information to be paid to the vehicle networking communication terminal; a vehicle networking communication terminal, located in the vehicle, in wireless communication with the mobile terminal, and configured to send the vehicle fee information to be paid to the mobile terminal; A mobile terminal, configured to perform a vehicle fee payment operation according to the vehicle fee information to be paid; Before the information collection device collects the vehicle fee information to be paid, the method further includes: When the control unit determines that there is a toll booth in front of the vehicle, a remote payment request is sent to the vehicle network communication terminal; sending the remote payment request to the mobile terminal via the Internet of Vehicles communication terminal, receiving instruction information returned by the mobile terminal, and transmitting the instruction information to the control unit; When the control unit determines that the instruction information is a remote payment instruction, the information collection device is triggered to collect information.

2. The system according to claim 1, wherein: The information acquisition device is an image acquisition device, and the image acquisition device is used to: An image including the vehicle fee information to be paid is captured and the image is transmitted to the control unit.

3. The system according to claim 2, characterized in that The image acquisition device is installed on the B-pillar of the vehicle.

4. The system according to claim 2 or 3, characterized in that In the image, the vehicle fee information is represented by a QR code.

5. The system according to claim 4, characterized in that The image acquisition device is further configured to: perform convolution processing on the image, locate the QR code region in the image, extract the QR code region in the image based on the location result, obtain a QR code image, and transmit the QR code image to the control unit; The control unit is specifically configured to: transmit the QR code image to the Internet of Vehicles terminal; The vehicle network terminal is specifically used to: send the QR code image to the mobile terminal; The mobile terminal is specifically used to: identify the two-dimensional code image, obtain vehicle fee information to be paid, and perform a fee payment operation according to the vehicle fee information to be paid.

6. The system according to claim 2, wherein: The control unit is connected to the vehicle networking communication terminal via a hard line.

7. A vehicle fee payment method, characterized in that: include: collecting vehicle fee information to be paid through an information collection device and transmitting the vehicle fee information to be paid to a control unit; The information collection device is installed on the main driver's side of the vehicle; transmitting the to-be-paid vehicle fee information to the Internet of Vehicles communication terminal via the control unit; The vehicle fee information to be paid is sent to the mobile terminal via the vehicle networking communication terminal, and the mobile terminal is used to perform a vehicle fee payment operation according to the vehicle fee information to be paid; Before the information collection device collects the vehicle fee information to be paid, the method further includes: When the control unit determines that there is a toll booth in front of the vehicle, a remote payment request is sent to the vehicle network communication terminal; sending the remote payment request to the mobile terminal via the Internet of Vehicles communication terminal, receiving instruction information returned by the mobile terminal, and transmitting the instruction information to the control unit; When the control unit determines that the instruction information is a remote payment instruction, the information collection device is triggered to collect information.

8. The method according to claim 7, characterized in that The information acquisition device is an image acquisition device; The method includes collecting vehicle fee information to be paid through an information collection device and transmitting the vehicle fee information to be paid to a control unit, including: An image including the vehicle fee information to be paid is captured by the image capturing device, and the image is transmitted to the control unit.

9. The method according to claim 8, characterized in that The method includes: collecting an image including the vehicle fee information to be paid by the image collecting device and transmitting the image to the control unit, including: Capturing an image including the vehicle fee information to be paid by the image acquisition device; performing convolution processing on the image using an image processing model by the image acquisition device, locating a two-dimensional code region in the image, extracting the two-dimensional code region in the image based on the positioning result, obtaining a two-dimensional code image, and transmitting the two-dimensional code image to the control unit; Transmitting the to-be-paid vehicle fee information to the vehicle networking communication terminal through the control unit includes: transmitting the QR code image to the Internet of Vehicles communication terminal through the control unit; The method includes sending the vehicle fee information to be paid to the mobile terminal through the Internet of Vehicles communication terminal, including: The QR code image is sent to the mobile terminal through the Internet of Vehicles communication terminal. The mobile terminal is used to identify the QR code image, obtain the vehicle fee information to be paid, and perform the fee payment operation according to the vehicle fee information to be paid.

10. The method according to claim 9, characterized in that Before the image acquisition device uses the image processing model to perform definition improvement processing on the image, the method further includes: clustering the images using a clustering model by the image acquisition device; The image acquisition device performs convolution processing on the image using an image processing model, including: If the clustering result indicates that the image is classified as a cluster including a two-dimensional code area, the image is convolutionally processed by the image acquisition device using an image processing model.

Images