Automatic tracking reading method in mileage verification process

Abstract

The invention provides an automatic tracking reading method in a mileage verification process, which comprises the following specific steps of: acquiring an image of a detection area, and analyzing the acquired image through a background difference algorithm to judge whether a large target enters the detection area or not; based on the analyzed image, searching a large target in an image field of view and a small target in the large target through a target search algorithm, and obtaining a clear small target image by amplifying and focusing the field of view where the small target is located; acquiring a template image of the position of the small target, marking the relative position of the mileage data in the template image, and calculating a theoretical value of the position of the mileage data of the small target in the actual attitude according to the relative position; based on the theoretical value, rectangular position information of the small target mileage data is extracted; analyzing current mileage data based on the rectangular position information of the small target mileage data; by means of the method, the mileage data in the small target can be accurately read, and convenience is provided for follow-up processing of the mileage data.

Description

Technical Field

[0001] The invention relates to a mileage measurement verification technology, and in particular to an automatic tracking reading method during a mileage verification process. Background Art

[0002] With the acceleration of urbanization, mileage data is used as one of the main detection parameters in many industries. It can provide various conditions corresponding to the current detection. For example, the mileage data in the dashboard of the vehicle can reflect the age and usage of the current vehicle. Another example is taxis. Taxis play an important role in the infrastructure of the city. As an important measuring instrument for operation and settlement, taxi meters play an important role in ensuring fair trade and consumer rights. The meter measures the rental time and mileage to calculate the fees that consumers should pay. Accurate and stable meters provide protection for the interests of both passengers and drivers. In addition, for example, the mileage on the display of a treadmill can reflect the distance and time of the current exercise of the athlete, providing convenience for the athlete.

[0003] However, in these application scenarios, most of the mileage detection methods are to obtain specific data through manual observation. In the manual observation method, such as reading the mileage of the taxi meter, the area that can be observed manually is small due to the installation location of the taxi meter and the size of the taxi window. In addition, the mileage number displayed by the taxi meter is small in size and has poor clarity, so the accuracy of recognition is low, which further increases the difficulty of meter verification. Summary of the invention

[0004] In order to overcome the deficiencies of the prior art, the present invention provides a method for automatically tracking and reading mileage during a mileage verification process.

[0005] In order to achieve the above-mentioned purpose, the present invention provides a method for automatically tracking and reading mileage during verification, and the specific steps include:

[0006] Collect images of the detection area and analyze them using a background difference algorithm to determine whether a large target has entered the detection area;

[0007] Based on the analyzed image, the target search algorithm is used to search for large targets in the image field of view and small targets within the large targets. By magnifying and focusing the field of view where the small targets are located, a clear image of the small targets is obtained.

[0008] Obtain a template image of the location of the small target, mark the relative position of the mileage data in the template image, and calculate the theoretical value of the location of the small target mileage data under the actual posture based on the relative position; based on the theoretical value, extract the rectangular position information of the small target mileage data; based on the rectangular position information of the small target mileage data, parse the current mileage data.

[0009] Preferably, the specific steps of the background difference algorithm include:

[0010] Set the current image as the background image and obtain the real-time current frame from the video stream;

[0011] Calculate the difference between the current frame and the background image, and compare pixel by pixel. The part where the pixel difference exceeds the preset threshold is regarded as the foreground object, and the area with small difference is regarded as the background. The calculation formula is:

[0012] D(x,y)=|I currcnt (x,y)-I background (x,y)|

[0013] Where D(x,y) is the difference between the current frame and the background frame at the coordinate (x,y); I current (x, y) is the pixel value of the current frame at the coordinate (x, y); background (x, y) is the pixel value of the background image at the same coordinate (x, y);

[0014] The calculated difference image is binarized, and the specific calculation formula is:

[0015]

[0016] Where Binary(x,y) is the binary pixel value at the coordinate (x,y); if D(x,y) is greater than the set threshold T, the output at the current pixel point (x,y) is 1, indicating that a foreground object exists; if D(x,y) is less than or equal to the set threshold T, the output at the current pixel point is 0, indicating that it belongs to the background;

[0017] After the calculation is completed, the number of pixels with a median value of 1 among all pixels is counted and compared with the total number of pixels. If the comparison value is greater than the first preset value, it is judged that a large target has entered. If the comparison value is less than the first preset value but greater than the second preset value, it is judged that other objects have entered. If the comparison value is less than the second preset value, it is judged that no object has entered.

[0018] Preferably, before searching for large targets and small targets, after detecting the targets using a positioning tracking algorithm, the targets are marked in the form of a marking frame, and a collection device is called to track the targets in real time. The specific steps include:

[0019] Coordinate transformation:

[0020] Based on the pixel coordinate system of the current view window, calculate the center position of the annotation box. The calculation formula is:

[0021]

[0022] Among them, centerX is the horizontal coordinate of the center of the annotation box; centerY is the vertical coordinate of the center of the annotation box; top-left is the horizontal coordinate of the top left corner of the annotation box; x bottom-right is the horizontal coordinate of the vertex of the lower right corner of the annotation box; y top-left is the vertical coordinate of the top left corner of the annotation box; y bottom-right The vertical coordinate of the vertex of the lower right corner of the annotation box;

[0023] Parameter settings:

[0024] Use CHCNetSDK.NET_DVR_POINT_FRAME structure to store location information to provide relevant parameters for acquisition device tracking;

[0025] Trace calls:

[0026] After the coordinate conversion is completed and stored, use the NET_DVR_PTZSelZoomIn_EX function to take the data stored in the CHCNetSDK.NET_DVR_POINT_FRAME structure as input to notify the acquisition device to perform positioning and tracking.

[0027] Preferably, the steps of locating large targets and small targets by the target search algorithm include:

[0028] Large target search steps:

[0029] Set multiple related parameters as the zero point position, return to zero after each inspection task is completed, and adjust the field of view size and focus parameters to the default values;

[0030] Identify whether there is a large target within the field of view of the current zero position. If there is no large target, no operation is performed and the information that the large target has not reached the specified position is returned;

[0031] Small target search steps:

[0032] If the small target can be directly identified in a large field of view environment, go directly to the next step;

[0033] If the small target cannot be directly identified under the large field of view, the image is partially enlarged based on the center of the image under the large field of view, and the enlarged area is detected again to see if there is a small target;

[0034] If the zoomed-in area still cannot identify the small target, the camera moves outward in a nine-square pattern with the current shooting position as the center and detects in sequence. If a small target is detected at one position, the camera stops moving and proceeds to the next step.

[0035] If the nine-grid search is completed but the small target is still not recognized, the message that the small target is not recognized is returned.

[0036] Preferably, the step of locating the large target and the small target by the target search algorithm further comprises:

[0037] Small target position correction steps:

[0038] If a small target has been identified, the posture and angle of the camera are adjusted according to the position of the small target to the center of the field of view, and the small target is controlled to move to the center of the field of view and parallel to the image;

[0039] Steps to zoom in on the target and refocus:

[0040] If the small target has moved to the center of the field of view, the detection target is enlarged and refocused. The specific steps include:

[0041] According to the area where the identified small target is located, the image field of view is enlarged according to the ratio of the length of its circumscribed rectangle to the length of the entire field of view. The image magnification ratio is calculated as follows:

[0042] Image long side magnification = (image long side * N) / small target long side,

[0043] Where N is the ratio of the long side of the small target to the long side of the image field of view after enlargement;

[0044] The clarity of the enlarged image is calculated using an image clarity evaluation algorithm, and the image focus parameters are adjusted based on the clarity.

[0045] Preferably, the specific execution steps of the image clarity evaluation algorithm include:

[0046] According to the position of the small target dial that has been identified in the target search process, adjust the camera to control the small target to rotate and translate to the center of the field of view;

[0047] Adjust the camera focus and zoom in on the image;

[0048] Calculate the position of the small target dial in the enlarged image and crop it out;

[0049] Based on the target detection results, the length and width of the small target before magnification are C1 and R1 respectively. After magnification, the ratio of the long side of the small target to the long side of the image field of view is N, and the length C2 and width R2 of the small target after magnification are obtained. The calculation formula is:

[0050] C2=Ic*N;

[0051] R2=C2 / C1*R1;

[0052] Among them, Ic is the image length, and the location of the small target in the enlarged image is in the rectangular area with the center point of the image as the center, C2 and R2 are the length and width respectively, and the calculation area is cropped out.

[0053] Preferably, the specific execution steps of the image clarity evaluation algorithm also include:

[0054] Calculate the edge clarity of the small target dial in the rectangular area; the calculation formula of clarity Var is:

[0055]

[0056] Where n is the number of pixels, x i represents the pixel value, Represents the average value of pixel values;

[0057] Determine whether the current sharpness is within the set threshold range. If so, end the focusing process, otherwise proceed to the next step;

[0058] Determine whether the current image is the first frame. If so, save the current definition value as a1. For the first time, set the lens focus direction to the positive direction by default. The focus moves with a step size of s. The step size s is set by the host computer. If the current image is not the first frame, proceed to the next step.

[0059] Determine whether the current clarity value is greater than the current clarity value a1. If it is greater than the current clarity value a1, continue to move once in the same direction by the step value s according to the last focus direction. If it is less than or equal to the current clarity value a1, move once in the opposite direction by the step value s. After the focus ring moves, return to the small target dial edge clarity step.

[0060] Preferably, the step of extracting the rectangular position information of the mileage data in the small target includes: a coarse positioning step:

[0061] Collect a small target template image, and mark the small target dial position and mileage position in the small target template image;

[0062] Obtain the rectangular area position information of the small target dial: center point (Jx, Jy), angle Ja, rectangle length and width (Jw, Jh) and kilometer data rectangle position information: center point (Gx, Gy), angle Ga, rectangle length and width (Gw, Gh);

[0063] Calculate the relative position relationship, the calculation formula is:

[0064] Center point deviation: x Δ =Gx-Jx,y Δ =Gy-Jy;

[0065] Angle deviation: a Δ =Ga-Ja;

[0066] Rectangle aspect ratio: w r =Gw / Jw,h r =Gh / Jh;

[0067] After the autofocus is completed, the position of the small target in the current image is obtained as follows: center point (nJx, nJy), angle nJa, length and width of the rectangle (nJw, nJh);

[0068] Based on the location of the small target, the theoretical values ​​of the center point (nGx, nGy) of the kilometer location, the angle nGa, the length and width of the rectangle (nGw, nGh) and the area RA are:

[0069] Center point: nGx=nJx+x Δ , nGy=nJy+y Δ ;

[0070] Angle: nGa=nJa*a Δ ;

[0071] Length and width: nGw = nJw*w r ; nGh=nJh*h r ;

[0072] Area: RA = nGw*nGh.

[0073] Preferably, the step of extracting the rectangular position information of the small target mileage data includes: a precise positioning step, specifically:

[0074] Get the cropped small target dial area;

[0075] Extract all character positions through OCR character recognition to obtain the rectangular area position information of all characters;

[0076] Filter out the mileage data area;

[0077] Determine whether the number of remaining areas after filtering is equal to 1. If so, return the location information of the current area. The current area is the area where the small target mileage data is located. If not, return the detection anomaly.

[0078] Preferably, after the rectangular position information of the small target mileage data is extracted, screening is performed within a certain deviation range based on the theoretical value obtained by coarse positioning.

[0079] Preferably, the precise positioning steps are:

[0080] Get the cropped small target dial area;

[0081] Binarize the obtained cropped area data and extract the highlighted digital tube area whose grayscale threshold is within the set range;

[0082] Perform closing operation with 11*11 element structure to connect adjacent digital tube areas;

[0083] According to the theoretical area value RA of the rectangular area of ​​the mileage data obtained by coarse positioning, the area of ​​the connected domain after the closing operation is screened within a certain range of area value intervals;

[0084] Extract all the connected domain contours filtered in the previous step, fit the minimum circumscribed rectangle of the contour, and calculate the coordinates of the center point of the minimum circumscribed rectangle, the length and width of the rectangle, and the angle of the rectangle;

[0085] Filter out the mileage data area;

[0086] Determine whether the number of remaining areas after filtering is equal to 1. If so, return the location information of the current area. The current area is the area where the mileage data of the small target is located. If not, return the detection anomaly.

[0087] Preferably, the reading method uses an OCR character recognition model to parse the current mileage data.

[0088] Preferably, the step of parsing the current mileage data using the OCR recognition model includes:

[0089] Set the minimum unit increment value a of the small goal, which means that the mileage data of the current small goal is accumulated by unit a each time starting from 0;

[0090] Collect and use the OCR character recognition model to parse the mileage value of the first frame image; determine whether the initial mileage value is 0, if so, proceed to the next step, if not, return an abnormal signal;

[0091] Record current mileage data;

[0092] Determine whether the mileage value read from the next frame of image is equal to the value of the previous frame of image, or is greater than the value of the previous frame of image by a; if it meets the set conditions, update the currently read value to the recorded mileage value; if it does not meet the set conditions, the current image frame is in the digital tube switching process, and proceed to the next step;

[0093] Determine whether the currently saved mileage value reaches the set value. If it reaches the set value, end the current process and return a completion signal. If it does not reach the set value, continue to capture the next frame of image.

[0094] The method for automatically tracking and reading the mileage verification process provided by the present invention has the following beneficial effects:

[0095] The present invention analyzes the collected image through an image background difference algorithm to determine whether a large target enters the detection area, providing a guarantee for further identifying and locating the small target in the large target. If a large target enters, the small target in the large target is located through a target search algorithm, and the clarity of the small target is processed to provide a guarantee for better identification of the mileage in the small target. For a clear small target image, a template image in the small target is collected and the position of the mileage data required for detection is marked in the template image. After the position of the small target is detected, the theoretical value of the position of the small target mileage data under the actual posture is converted according to the relative position, so that the detection is more universal. Based on the theoretical value, the rectangular position information of the key characters in the small target dial is extracted to overcome the deviation generated during deep learning target detection during actual recognition. Based on the rectangular position information, the OCR character recognition model is used to parse the current mileage data displayed on the digital tube, and the stroboscopic problem of the digital tube when switching data is also overcome. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Figure 1 A flow chart of the method for automatically tracking and reading the mileage verification process provided by the present invention;

[0097] Figure 2 A flowchart of performing target search in an embodiment of the present invention;

[0098] Figure 3 A schematic diagram of identifying a small field of view in a large field of view of a large target when performing target search in an embodiment of the present invention;

[0099] Figure 4 A schematic diagram of the position of a small target located within a large target when performing target search in an embodiment of the present invention;

[0100] Figure 5 This is a schematic diagram of performing a nine-square grid search when performing a target search in an embodiment of the present invention;

[0101] Figure 6 A schematic diagram of correcting the position of a small target when performing target search in an embodiment of the present invention;

[0102] Figure 7 A schematic diagram of magnifying and focusing a target image when performing target search in an embodiment of the present invention;

[0103] Figure 8 A schematic diagram of adjusting camera acquisition in an embodiment of the present invention;

[0104] Fig. 9 A schematic diagram of an image of a small target magnified in an embodiment of the present invention;

[0105] Fig.10A schematic diagram showing that an image of a small target magnified in an embodiment of the present invention changes from blurry to clear;

[0106] Fig.11 is a flow chart of an automatic focusing process in an embodiment of the present invention;

[0107] Fig.12 A flowchart of the mileage data positioning of the taximeter in an embodiment of the present invention;

[0108] Fig.13 A schematic diagram of a taximeter that needs to be identified and the contents displayed in the taximeter in an embodiment of the present invention;

[0109] Fig.14 A schematic diagram of a taximeter template image obtained in an embodiment of the present invention;

[0110] Fig.15 A schematic diagram of obtaining rectangular area position information of all characters in an embodiment of the present invention;

[0111] Fig.16 It is a schematic diagram of filtering out the area where the required taximeter mileage data is located from the rectangular area position information of all characters in an embodiment of the present invention;

[0112] Fig.17 This is a flow chart of parsing mileage data in an embodiment of the present invention;

[0113] Fig.18 It is a schematic diagram of the flicker problem when the digital tube switches data in the embodiment of the present invention. DETAILED DESCRIPTION

[0114] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0115] like Figure 1 As shown, the method for automatically tracking and reading the mileage verification process provided by the present invention specifically comprises the following steps:

[0116] Collect images of the detection area and analyze them using a background difference algorithm to determine whether a large target has entered the detection area;

[0117] Based on the analyzed image, the target search algorithm is used to search for large targets in the image field of view and small targets within the large targets. By magnifying and focusing the field of view where the small targets are located, a clear image of the small targets is obtained.

[0118] Obtain a template image of the location of the small target, mark the relative position of the mileage data in the template image, and calculate the theoretical value of the location of the small target mileage data under the actual posture based on the relative position; based on the theoretical value, extract the rectangular position information of the small target mileage data; based on the rectangular position information of the small target mileage data, parse the current mileage data.

[0119] The present invention can be applied to a variety of scenarios, such as mileage on a car dashboard, mileage recognition on a treadmill dial, etc. It only needs to locate the energy efficiency target in the large target and then recognize the mileage characters from the small target.

[0120] In order to better explain the specific mileage reading method of the present invention, the present invention is described by taking the application scenario of reading the mileage displayed on the meter as an example. In the application scenario of reading the mileage of the meter, the vehicle (such as a taxi) is a large target among the detection objects, and the meter is a small target among the detection objects. By detecting the vehicle, the position of the meter in the vehicle is located, and the mileage characters displayed on the meter are identified based on the meter position. The details are as follows:

[0121] In this embodiment, the collected image is analyzed by the image background difference algorithm to determine whether there is a vehicle entering the detection area, which provides a guarantee for further identifying and locating the meter in the vehicle. If a vehicle enters, the meter in the vehicle is located by the target search algorithm, and the clarity of the meter is processed to provide a guarantee for better identification of the meter mileage. For a clear meter image, a template image in the meter is collected and the position of the mileage data required for detection is marked in the template image. After the meter position is detected, the theoretical value of the position of the meter mileage data under the actual posture is converted according to the relative position, so that the detection is more universal. Based on the theoretical value, the rectangular position information of the key characters in the meter display screen is extracted to overcome the deviation generated by deep learning target detection during actual recognition. Based on the rectangular position information, the OCR character recognition model is used to parse the current mileage data displayed on the digital tube, and the stroboscopic problem of the digital tube when switching data is also overcome.

[0122] In the present invention, the core idea of ​​the background difference algorithm is: in an image, if the background is static, by comparing it with a saved background image, the changed pixels in the current image are marked as foreground objects, that is, moving objects; and the unchanged pixels are considered to belong to the background.

[0123] The process of background difference algorithm includes:

[0124] Background Modeling: At startup or within a specific time period, one or more images are captured to build a background model. The background image is usually a state where there are no moving objects in the scene.

[0125] Background Update: Over time, the scene may undergo small changes, such as changes in lighting conditions, so the background model needs to be dynamically updated to reflect these changes in the scene.

[0126] Foreground Extraction: Extract foreground objects (moving objects) by comparing the differences between the current image and the background model.

[0127] The core goal of the background difference algorithm used in the present invention is to detect the foreground object (i.e., the vehicle) when the vehicle enters the detection area, and to distinguish the background from the foreground by background difference. The detected background is a scene without vehicles and interference. In the initial stage, one or more background images are obtained by a shooting device (in this embodiment, a ball camera is used to shoot images) and used as a reference. The new frame image obtained by the subsequent ball camera is compared with the background image. If some areas in the image have changed significantly (such as the entry of a vehicle), the changed area will be identified as the foreground, that is, the area where the vehicle is located.

[0128] In the present invention, the specific steps of the background difference algorithm include:

[0129] Set the current image as the background image and get the real-time current frame from the video stream.

[0130] Calculate the difference between the current frame and the background image, and compare pixel by pixel. For the part where the pixel difference exceeds the preset threshold, it is regarded as the foreground object, and the area with smaller difference is regarded as the background. The formula can be expressed as:

[0131] D(x,y)=|I currcnt (x,y)-I background (x,y)|

[0132] Where D(x,y) is the difference between the current frame and the background frame at the coordinate (x,y); I current (x, y) is the pixel value of the current frame at the coordinate (x, y); background (x,y) is the pixel value of the background image at the same coordinate (x,y).

[0133] After the difference algorithm is completed, in order to further distinguish the foreground object from the background, it is usually necessary to perform binarization on the difference image. That is, pixels greater than a certain threshold are marked as foreground (1), and pixels less than the threshold are marked as background (0). This process is completed by setting the threshold T, and its specific formula is expressed as:

[0134]

[0135] Among them, Binary(x,y) is the binary pixel value at the coordinate (x,y);

[0136] If D(x,y) is the difference between the previous frame and the background frame at the pixel (x,y), if D(x,y) is greater than the set threshold T, the output at the pixel (x,y) is 1 (indicating the presence of a foreground object). If D(x,y) is less than or equal to the set threshold T, the output at the pixel is 0 (indicating that it belongs to the background). After the calculation is completed, the number of pixels with a median value of 1 (foreground object) is counted among all pixels and compared with the total number of pixels (i.e., background pixels). If it is greater than 60%, it is judged that a vehicle has entered; if it is less than 60% and greater than 10%, it is judged that other objects have entered; if it is less than 10%, it is judged that no object has entered.

[0137] After obtaining the binary image, the binary image is post-processed to improve the detection accuracy of the foreground object. The post-processing operation includes: using morphological processing (such as corrosion) to remove noise, and then using connected domain analysis to extract the complete foreground object. In the final foreground image, the white area represents the moving foreground object, and the black area represents the static background. The result can be further used for analysis and processing, such as triggering the detection of the taximeter.

[0138] After the vehicle is identified, the positioning and tracking algorithm is used to detect the target and mark the target in the form of a bounding box. The algorithm calculates the relative coordinates of the target based on the center position in the bounding box, and then calls the PTZ camera for real-time tracking. The steps include:

[0139] I: Coordinate transformation steps:

[0140] When the model returns the annotation box, the center position of the annotation box is calculated, which is based on the pixel coordinate system of the current view window. In order for the PTZ camera to accurately track the position, the coordinates of the center point of the annotation box are converted to a coordinate system that the PTZ camera can understand. The PTZ camera uses a normalized coordinate system, which usually ranges from 0 to 255. Therefore, this step requires converting the pixel coordinates in the current view window to the coordinate system of the PTZ camera. The center point of the annotation box is calculated by calculating the x and y coordinates of the center through the vertex coordinates of the annotation box. The calculation formula is:

[0141]

[0142] Among them, centerX is the horizontal coordinate of the center of the annotation box; centerY is the vertical coordinate of the center of the annotation box; top-left is the horizontal coordinate of the top left corner of the annotation box; x bottom-right is the horizontal coordinate of the vertex of the lower right corner of the annotation box; y top-left is the vertical coordinate of the top left corner of the annotation box; y bottom-right The vertical coordinate of the lower right corner of the annotation box.

[0143] II: Parameter settings:

[0144] After the calculation is completed, the coordinate position of the moving target is obtained. After the coordinate conversion is completed, the CHCNetSDK.NET_DVR_POINT_FRAME structure is used to store the position information of the ordinate and abscissa to provide the relevant parameters for ball camera tracking, namely:

[0145] xTop: The horizontal coordinate of the top of the target (X-axis direction). The horizontal position of the target object in the upper left corner of the image. yTop: The vertical coordinate of the top of the target (Y-axis direction). The vertical position of the target object in the upper left corner of the image. xBottom: The horizontal coordinate of the bottom of the target (X-axis direction). The horizontal position of the target object in the lower right corner of the image. yBottom: The vertical coordinate of the bottom of the target (Y-axis direction). The vertical position of the target object in the lower right corner of the image. bCounter: Tracking counter, used to record or control the number of tracking times, which can help monitor the tracking status or be used to determine the effectiveness of the tracking target.

[0146] III: Tracking calls:

[0147] After the coordinate conversion is completed and stored, use the NET_DVR_PTZSelZoomIn_EX (lateral and longitudinal movement of the ball camera) function to input the data stored in the CHCNetSDK.NET_DVR_POINT_FRAME structure to notify the ball camera to locate and track. This function will accept the converted coordinate parameters and pass them to the control logic of the ball camera, instructing the ball camera to start tracking the specified position.

[0148] like Figure 2 As shown, based on the image after background analysis, searching for vehicles in the image and the meter in the vehicle includes the following steps:

[0149] (1) Vehicle search steps:

[0150] Set a detection position and the default field of view size and focus and other related parameters as the zero point position. The default field of view size is a large field of view that can see the taxi from the front to more than half of the body. After each detection task is completed, it is necessary to return to zero and adjust the field of view size and focus parameters to the default values;

[0151] Identify whether there is a vehicle within the field of view of the current zero position. If there is no vehicle, no other operations are performed and the information that the vehicle has not reached the specified position is returned.

[0152] (2) The meter search step includes two situations, namely, direct recognition and non-recognition. Specifically:

[0153] If the meter can be directly identified in a large field of view, go directly to the next step; Figure 3 , 4 shown.

[0154] If the taximeter cannot be directly identified under the large field of view, the image is partially enlarged based on the center of the image under the large field of view, and then the enlarged area is detected again to see if there is a taximeter;

[0155] If the taximeter is still not recognized in the zoomed-in area, the camera moves outward in a nine-square pattern with the current camera shooting position as the center and detects in sequence. As long as a taximeter is detected in one position, the camera stops moving and proceeds to the next step.

[0156] If the meter is not recognized after the nine-grid search is completed, it means that the meter may be blocked or not installed, and the message "the meter is not recognized" will be returned. Figure 5 shown.

[0157] (3) Meter position correction steps:

[0158] If the taximeter is recognized, the camera posture and angle are further adjusted according to the position of the taximeter to the center of the field of view, so that the taximeter moves to the center of the field of view and is parallel to the image. Figure 6 shown.

[0159] (4) Magnification and refocusing of the detection target:

[0160] If the taximeter has moved to the center of the field of view, zoom in on the detection target and refocus. The specific steps include:

[0161] According to the area where the taximeter is identified, the image field of view is enlarged according to the ratio of the length of its circumscribed rectangle to the length of the entire field of view. The image magnification ratio is calculated as follows:

[0162] Image long side magnification = (image long side * N) / meter long side,

[0163] Wherein, N is the ratio of the long side of the enlarged taximeter to the long side of the image field of view. If the enlarged taximeter needs to occupy more than 70% of the field of view, N=0.7.

[0164] After zooming in, the image will become blurred. The image clarity is then calculated using the image clarity evaluation algorithm. The image focus parameters are adjusted based on the clarity to make the detection target clear. Figures 7-10 shown.

[0165] In order to ensure the accuracy of character recognition and the clarity of the image, it is necessary to adjust the lens focal length to magnify the image until it is refocused and clear. Since the general clarity detection algorithm calculates the clarity of the entire image, it is difficult to capture all the subtle changes in image clarity, especially when the image clarity changes are not very obvious. Therefore, the clarity of the entire image cannot be calculated, but the clarity of the meter display screen area is calculated to achieve the best focus effect.

[0166] In the present invention, the working principle of the image clarity evaluation algorithm is as follows: based on the identified rectangular area of ​​the taximeter display screen, the display area is first cropped out, and then the image sharpness is calculated based on the sharpness change of the edge contour of the display, which is used as the image clarity, and then the camera focus is adjusted according to the clarity. The camera itself has an electronic focus ring, and the moving step of the electronic focus ring can be manually set by the host computer. The entire process of focusing is a cyclic process of focusing-evaluation until the clarity reaches the manually set threshold range.

[0167] The specific execution steps of the image clarity evaluation algorithm are:

[0168] S1: According to the target search process, the position of the taximeter display screen has been identified, and the camera is adjusted so that the taximeter is rotated and translated to the center of the field of view and the posture of the taximeter is basically parallel to the image coordinates.

[0169] S2; adjust the camera focus and enlarge the image.

[0170] S3: Calculate the position of the taximeter display screen in the enlarged image and crop it out.

[0171] From the target detection results, we can get the length and width of the taximeter before magnification as C1 and R1 respectively, and the ratio N of the long side of the taximeter to the long side of the image field of view after magnification can get the length and width C2 and R2 of the taximeter after magnification. The specific calculation formula is:

[0172] C2=Ic*N,

[0173] R2=C2 / C1*R1,

[0174] Where Ic is the length of the image. The location of the taximeter in the enlarged image is in the rectangular area with the center of the image as the center, C2 and R2 as the length and width respectively, and the area is cropped out.

[0175] S4: Calculate the edge clarity of the meter display screen in the rectangular area. The calculation formula of clarity Var is:

[0176]

[0177] Where n is the number of pixels, x i represents the pixel value, Indicates the average value of pixel values. The larger the calculated value is, the greater the grayscale change of the image is, that is, the higher the clarity is.

[0178] S5: Determine whether the current sharpness is within the set threshold range, if yes, end the focusing process, otherwise proceed to the next step.

[0179] S6: Determine whether the current image is the first frame. If so, save the current clarity value as a1. For the first time, the lens focus direction is set to the positive direction by default. The focus moves with a step size of s. The step size s can be set by the host computer. If the current image is not the first frame, proceed to the next step.

[0180] S7: Determine whether the current clarity value is greater than a1. If it is greater than a1, continue to move once in the same direction according to the last focus direction by the step value s. If it is not greater than a1, move once in the opposite direction by the step value s. After the focus ring moves, return to step S4. The auto focus process flow chart is as follows Fig.11 shown.

[0181] In the present invention, after obtaining the enlarged field of view of the location of the taximeter, the mileage of the taximeter is located, such as Fig.12 As shown, the steps include:

[0182] (1) Coarse positioning:

[0183] Since the display content positions of meters produced by different manufacturers are all different, and the placement posture of the meter is slightly different due to the actual position of the taxi, when performing coarse positioning, it first collects the template image and marks the position of the mileage data to be detected in the template image. After the meter position is detected, the theoretical value of the position of the meter mileage data under the actual posture can be converted according to the relative position, making the detection more universal.

[0184] The specific positioning steps include:

[0185] ① The host computer collects the meter template map and manually marks the meter display screen position and mileage (kilometer) position; Fig.13 ,14 shown.

[0186] The rectangular area position information of the meter display screen can be obtained as follows: center point (Jx, Jy), angle Ja, length and width of the rectangle (Jw, Jh);

[0187] And the kilometer data rectangle position information: center point (Gx, Gy), angle Ga, rectangle length and width (Gw, Gh) Based on the above information, calculate the relative position relationship:

[0188] Center point deviation: x Δ =Gx-Jx,y Δ =Gy-Jy;

[0189] Angle deviation: a Δ =Ga-Ja;

[0190] Rectangle aspect ratio: w r =Gw / Jw,h r =Gh / Jh.

[0191] ② After the detection process is completed with automatic focus at a specific position (i.e. the process after the ball camera enters automatic focus), the position of the taximeter in the current image can be obtained as follows: center point (nJx, nJy), angle nJa, length and width of the rectangle (nJw, nJh);

[0192] Based on the meter location, the theoretical values ​​of the center point of the kilometer location (nGx, nGy), angle nGa, length and width of the rectangle (nGw, nGh) and area RA are:

[0193] Center point: nGx=nJx+x Δ , nGy=nJy+y Δ ;

[0194] Angle: nGa=nJa*a Δ ;

[0195] Length and width: nGw = nJw*w r ; nGh=nJh*h r ;

[0196] Area: RA = nGw*nGh.

[0197] (2) Precision positioning:

[0198] Since deep learning target detection has certain deviations during actual recognition, the center point, angle, length and width are not completely equal to the values ​​calculated above. Therefore, in order to accurately locate the target, it is also necessary to extract the rectangular position information of the key characters in the meter display screen, and screen it within a certain deviation range based on the theoretical values ​​converted during rough positioning.

[0199] Among them, in one embodiment of the present invention, a deep learning OCR character recognition model is used to extract a character rectangular area, and the specific steps include:

[0200] Get the meter display screen area cropped by coarse positioning;

[0201] Through deep learning OCR character model recognition, all character positions are extracted to obtain the rectangular area position information of all characters; Fig.15 The specific extraction steps include:

[0202] Collect character sample image data, and use annotation tools to annotate character positions and corresponding categories;

[0203] Use the pre-training framework to initialize the model, read and split the data set, and pre-process the data set;

[0204] Build a training environment, set the model and training hyperparameters to train the model;

[0205] Evaluate the training model and check whether the accuracy of the training model meets expectations. If not, adjust the training parameters or data set. If it meets expectations, derive the recognition model.

[0206] The exported recognition model is used to infer the new image, extract the characters of the new image, and obtain the rectangular area position information of all characters.

[0207] Filter out the mileage data area and remove unnecessary interference areas. The screening method is as follows:

[0208] The theoretical value of the center point of the area where the mileage data is located (nGx, nGy), the theoretical value of the area angle nGa, the theoretical value of the area length and width (nGw, nGh), and the theoretical value of the area RA are obtained by coarse positioning. The following parameter deviations are set in advance by the host computer:

[0209] The center point x-coordinate deviation is Xd, the center point y-coordinate deviation is Yd, the angle deviation is Ad, the region length deviation is Wd, the region width deviation is Hd, and the region area deviation is RAd.

[0210] Filter all character regions obtained according to the following conditions:

[0211] The x-coordinate value of the center point of the region conforms to the interval [nGx-Xd,nGx+Xd];

[0212] The y coordinate value of the center point of the region conforms to the interval [nGy-Yd,nGy+Yd];

[0213] The regional angle value is within the interval [nGa-Ad, nGa+Ad];

[0214] The region length value is within the interval [nGw-Wd,nGw+Wd];

[0215] The area width value conforms to the interval [nGh-Hd,nGh+Hd];

[0216] The area value of the region fits within the interval [RA-RAd,RA+RAd].

[0217] The purpose of the location screening of the present invention is to locate the mileage data, or the value corresponding to "kilometers", through parameters such as area size and center position. Fig.16 The data in the red rectangle above the "kilometer" character is shown. There is only one set of mileage data in any meter, so it is normal when the number of areas after the final screening is equal to 1, and other values ​​are abnormal.

[0218] In another embodiment of the present invention, an image preprocessing method is used to extract a character rectangular area, and the specific steps include:

[0219] Get the cropped meter display area;

[0220] Binarize the obtained cropped area data and extract the highlighted digital tube area with a grayscale threshold between 128 and 255;

[0221] Perform closing operation with 11*11 element structure to connect adjacent digital tube areas;

[0222] According to the theoretical area value RA of the rectangular area of ​​mileage data obtained by coarse positioning, the area of ​​the connected domain after the closed operation is screened within the range of [RA*0.75, RA*1.25]. In this process, the purpose of using two selections is to extract the rectangular area of ​​mileage data. Because this method is not based on deep learning for recognition, the area, length, width and other dimensions of the mileage area converted during coarse positioning, and the center point position are used as features to filter out all those that do not meet the theoretical values, and finally only retain a rectangular area of ​​mileage data. At the same time, because the theoretical value converted by coarse positioning has deviations and will not be exactly equal to the size of the actual mileage data area, a certain range of area is set for screening. The values ​​of 0.75 and 1.25 can be set according to the actual application scenario.

[0223] Extract all the connected domain contours filtered in the previous step, fit the minimum bounding rectangle of the contour, and calculate the coordinates of the center point of the minimum bounding rectangle, the length and width of the rectangle, and the angle of the rectangle. The steps for fitting the minimum bounding rectangle are:

[0224] a. Calculate the center of mass:

[0225] Centroid is the average position of all points and can be calculated using the following formula:

[0226]

[0227] Among them, (x i ,y i ) are the coordinates of the ith point, and n is the total number of points.

[0228] b. Construct the covariance matrix:

[0229] The covariance matrix is ​​used to find the main direction of the point set, and its formula is:

[0230]

[0231] Among them, σ xx is the variance in the x direction, σ yy is the variance in the y direction, σ xy and σ yx is the covariance between x and y.

[0232] c. Calculate the eigenvalues ​​and eigenvectors of the covariance matrix:

[0233] The eigenvalues ​​and eigenvectors can be obtained by solving the following characteristic equation:

[0234]

[0235] This will give two eigenvalues ​​λ1 and λ2, and the corresponding eigenvectors v1 and v2. The eigenvectors corresponding to the larger eigenvalues ​​represent the principal directions of the point set.

[0236] d. Determine the width and height of the minimum enclosing rectangle:

[0237] Use the eigenvalues ​​to determine the width and height of the minimum bounding rectangle:

[0238]

[0239] Where w is the width of the rectangle, l is the length of the rectangle, is the square of the covariance between the x and y coordinates.

[0240] e. Calculate the rotation angle:

[0241] The rotation angle θ can be calculated from the slope of the eigenvector:

[0242]

[0243] Filter out the mileage data area and remove unnecessary interference areas; the screening method is as follows:

[0244] The theoretical value of the center point of the area where the mileage data is located (nGx, nGy), the theoretical value of the area angle nGa, the theoretical value of the area length and width (nGw, nGh), and the theoretical value of the area RA are obtained by coarse positioning. The following parameter deviations are set in advance by the host computer:

[0245] The center point x-coordinate deviation is Xd, the center point y-coordinate deviation is Yd, the angle deviation is Ad, the region length deviation is Wd, the region width deviation is Hd, and the region area deviation is RAd.

[0246] Filter all character regions obtained in step ⑤ according to the following conditions:

[0247] The x-coordinate value of the center point of the region conforms to the interval [nGx-Xd,nGx+Xd];

[0248] The y coordinate value of the center point of the region conforms to the interval [nGy-Yd,nGy+Yd];

[0249] The regional angle value is within the interval [nGa-Ad, nGa+Ad];

[0250] The region length value is within the interval [nGw-Wd,nGw+Wd];

[0251] The area width value conforms to the interval [nGh-Hd,nGh+Hd];

[0252] The area value of the region fits within the interval [RA-RAd,RA+RAd].

[0253] Determine whether the number of remaining areas after filtering is equal to 1. If so, return the location information of the current area. The current area is the area where the required meter mileage data is located. Otherwise, return a detection exception.

[0254] In the present invention, Fig.17 As shown. Based on the rectangular position of the acquired meter mileage data, the deep learning OCR character recognition model is used for parsing, and the currently displayed mileage value can be obtained after parsing. According to user needs, the mileage value starts counting from 0, and when a set value is reached, the counting needs to be stopped and a signal needs to be returned.

[0255] Among them, for the digital tube data on the meter display screen, when the digital tube switches data, due to its special stroboscopic mechanism, such as Fig.18As shown, when switching from number 1 to number 2, the digits that need to be displayed in number 2 will be lit first and the digital tubes that are not needed for number 1 will be turned off. However, under high frame rate image capture, it is easy to capture the moment when the digital tube in the lower right corner of number 1 has not been turned off in time, resulting in the formation of number 6 in the middle process, causing the digits in the middle process to be recognized, so the read data is inaccurate.

[0256] Based on the above problems, the specific steps for parsing the meter mileage data include:

[0257] ① The upper computer sets the minimum unit increment value a of the meter, which means that the mileage data of the current meter is accumulated in unit a starting from 0 (the increment value of different meters may be different, some are 0.1 kilometers, and some are 0.01 kilometers);

[0258] ② Collect and use the deep learning OCR character recognition model to parse the mileage value of the first frame image to determine whether the initial mileage value is 0. If so, enter the subsequent process. If not, it means that the initial value of the taximeter is abnormal and returns an abnormal signal;

[0259] ③Record current mileage data;

[0260] ④ Then determine whether the mileage value read from the next frame is equal to the value of the previous frame or is greater than the value of the previous frame by a. If this condition is met, the current read value is updated to the recorded mileage value; if this condition is not met, it is considered that the current image frame is in the digital tube switching process, the mileage data is not updated, and the next step is executed;

[0261] ⑤ Determine whether the currently saved mileage value reaches the set value of the host computer. If it reaches the set value, end the current process and return a completion signal. If it does not reach the set value of the host computer, continue to collect the next frame of image.

[0262] Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

Claims

1. The method for automatically tracking and reading the mileage verification process is characterized by: The specific steps include: Collect images of the detection area and analyze them using a background difference algorithm to determine whether a large target has entered the detection area; Based on the analyzed image, the target search algorithm is used to search for large targets in the image field of view and small targets within the large targets. By magnifying and focusing the field of view where the small targets are located, a clear image of the small targets is obtained. Obtain a template image of the location of the small target, mark the relative position of the mileage data in the template image, and calculate the theoretical value of the location of the small target mileage data under the actual posture based on the relative position; based on the theoretical value, extract the rectangular position information of the small target mileage data; based on the rectangular position information of the small target mileage data, parse the current mileage data.

2. The method for automatically tracking and reading the mileage verification process according to claim 1, characterized in that: The specific steps of the background difference algorithm include: Set the current image as the background image and obtain the real-time current frame from the video stream; Calculate the difference between the current frame and the background image, and compare pixel by pixel. The part where the pixel difference exceeds the preset threshold is regarded as the foreground object, and the area with small difference is regarded as the background. The calculation formula is: D(x,y)=|I current (x,y)-I background (x,y)|, Where D(x,y) is the difference between the current frame and the background frame at the coordinate (x,y); I current (x, y) is the pixel value of the current frame at the coordinate (x, y); background (x, y) is the pixel value of the background image at the same coordinate (x, y); The calculated difference image is binarized, and the specific calculation formula is: Where Binary(x,y) is the binary pixel value at the coordinate (x,y); if D(x,y) is greater than the set threshold T, the output at the current pixel point (x,y) is 1, indicating that a foreground object exists; if D(x,y) is less than or equal to the set threshold T, the output at the current pixel point is 0, indicating that it belongs to the background; After the calculation is completed, the number of pixels with a median value of 1 among all pixels is counted and compared with the total number of pixels. If the comparison value is greater than the first preset value, it is judged that a large target has entered. If the comparison value is less than the first preset value but greater than the second preset value, it is judged that other objects have entered. If the comparison value is less than the second preset value, it is judged that no object has entered.

3. The method for automatically tracking and reading the mileage verification process according to claim 1, characterized in that: Before searching for large and small targets, the positioning and tracking algorithm is used to detect the target and mark the target in the form of a marking box, and the acquisition device is called to track the target in real time. The specific steps include: Coordinate transformation: Based on the pixel coordinate system of the current view window, calculate the center position of the annotation box. The calculation formula is: Among them, centerX is the horizontal coordinate of the center of the annotation box; centerY is the vertical coordinate of the center of the annotation box; top-left is the horizontal coordinate of the top left corner of the annotation box; x bottom-right is the horizontal coordinate of the vertex of the lower right corner of the annotation box; y top-left is the vertical coordinate of the top left corner of the annotation box; y bottom-right The vertical coordinate of the vertex of the lower right corner of the annotation box; Parameter settings: Use CHCNetSDK.NET_DVR_POINT_FRAME structure to store the position information of ordinate and abscissa to provide relevant parameters for acquisition device tracking; Trace calls: After the coordinate conversion is completed and stored, use the NET_DVR_PTZSelZoomIn_EX function to take the data stored in the CHCNetSDK.NET_DVR_POINT_FRAME structure as input to notify the acquisition device to perform positioning and tracking.

4. The method for automatically tracking and reading the mileage verification process according to claim 1, characterized in that: The steps of locating large and small targets by the target search algorithm include: Large target search steps: Set multiple related parameters as the zero point position, return to zero after each inspection task is completed, and adjust the field of view size and focus parameters to the default values; Identify whether there is a large target within the field of view of the current zero position. If there is no large target, no operation is performed and the information that the large target has not reached the specified position is returned; Small target search steps: If the small target can be directly identified in a large field of view environment, go directly to the next step; If the small target cannot be directly identified under the large field of view, the image is partially enlarged based on the center of the image under the large field of view, and the enlarged area is detected again to see if there is a small target; If the zoomed-in area still cannot identify the small target, the camera moves outward in a nine-square pattern with the current shooting position as the center and detects in sequence. If a small target is detected at one position, the camera stops moving and proceeds to the next step. If the nine-grid search is completed but the small target is still not recognized, the message that the small target is not recognized is returned.

5. The method for automatically tracking and reading the mileage verification process according to claim 1, characterized in that: The steps of extracting the rectangular position information of the mileage data in the small target include: coarse positioning step: Collect a small target template image, and mark the small target dial position and mileage position in the small target template image; Obtain the rectangular area position information of the small target dial: center point (Jx, Jy), angle Ja, rectangle length and width (Jw, Jh) and kilometer data rectangle position information: center point (Gx, Gy), angle Ga, rectangle length and width (Gw, Gh); Calculate the relative position relationship, the calculation formula is: Center point deviation: x Δ =Gx-Jx,y Δ =Gy-Jy; Angle deviation: a Δ = Ga-Ja; Rectangle aspect ratio: w r =Gw / Jw,h r =Gh / Jh; After the autofocus is completed, the position of the small target in the current image is obtained as follows: center point (nJx, nJy), angle nJa, length and width of the rectangle (nJw, nJh); Based on the location of the small target, the theoretical values ​​of the center point (nGx, nGy) of the kilometer location, the angle nGa, the length and width of the rectangle (nGw, nGh) and the area RA are: Center point: nGx=nJx+x Δ , nGy=nJy+y Δ ; Angle: nGa = nJa * a Δ ; Length and width: nGw = nJw * w r ; nGh = nJh * h r ; Area: RA = nGw*nGh.

6. The method for automatically tracking and reading the mileage verification process according to claim 5, characterized in that: The steps of extracting the rectangular position information of the small target mileage data include: a precise positioning step, specifically: Get the cropped small target dial area; Extract all character positions through OCR character recognition to obtain the rectangular area position information of all characters; Filter out the mileage data area; Determine whether the number of remaining areas after filtering is equal to 1. If so, return the location information of the current area. The current area is the area where the small target mileage data is located. If not, return the detection anomaly.

7. The method for automatically tracking and reading mileage during verification according to claim 6, characterized in that: After extracting the rectangular position information of the small target mileage data, it is screened within a certain deviation range based on the theoretical value obtained by coarse positioning.

8. The method for automatically tracking and reading mileage during verification according to claim 6, characterized in that: The precise positioning steps are: Get the cropped small target dial area; Binarize the obtained cropped area data and extract the highlighted digital tube area whose grayscale threshold is within the set range; Perform closing operation with 11*11 element structure to connect adjacent digital tube areas; According to the theoretical area value RA of the rectangular area of ​​the mileage data obtained by coarse positioning, the area of ​​the connected domain after the closing operation is screened within a certain range of area value intervals; Extract all the connected domain contours filtered in the previous step, fit the minimum circumscribed rectangle of the contour, and calculate the coordinates of the center point of the minimum circumscribed rectangle, the length and width of the rectangle, and the angle of the rectangle; Filter out the mileage data area; Determine whether the number of remaining areas after filtering is equal to 1. If so, return the location information of the current area. The current area is the area where the mileage data of the small target is located. If not, return the detection anomaly.

9. The method for automatically tracking and reading mileage during mileage verification according to claim 1, characterized in that: This reading method uses an OCR character recognition model to parse out the current mileage data.

10. The method for automatically tracking and reading mileage during verification process according to claim 1, characterized in that: The steps of parsing the current mileage data by the OCR recognition model include: Set the minimum unit increment value a of the small goal, which means that the mileage data of the current small goal is accumulated by unit a each time starting from 0; Collect and use the OCR character recognition model to parse the mileage value of the first frame image; determine whether the initial mileage value is 0, if so, proceed to the next step, if not, return an abnormal signal; Record current mileage data; Determine whether the mileage value read from the next frame of image is equal to the value of the previous frame of image, or is greater than the value of the previous frame of image by a; if it meets the set conditions, update the currently read value to the recorded mileage value; if it does not meet the set conditions, the current image frame is in the digital tube switching process, and proceed to the next step; Determine whether the currently saved mileage value reaches the set value. If it reaches the set value, end the current process and return a completion signal. If it does not reach the set value, continue to capture the next frame of image.

Images