Method and system for adaptively reading vehicle identification code

Abstract

The invention provides a method and system for self-adaptively reading a vehicle identification code, and relates to the field of intelligent traffic, the method comprises the following steps: acquiring multi-mode sensing data of a vehicle entering a target area, performing three-dimensional reconstruction from image data acquired from a plurality of visual angles through stereo matching, generating a stereo model of the target area, and obtaining a three-dimensional model of the target area; according to the multi-modal sensing data, constructing a spatial position and a form of a vehicle in a three-dimensional model, and determining an area where a vehicle identification code is located; according to the three-dimensional reconstructed stereoscopic model and the spatial position and form of the vehicle in the stereoscopic model, the optimal viewing angle for reading the vehicle identification code is analyzed, and the angle and distance of a camera are adaptively adjusted to capture a target image containing a vehicle identification code area; and carrying out character recognition on the extracted vehicle identification code area, carrying out encryption and evidence storage on successfully read vehicle identification code character information through a block chain, and generating a vehicle identification code reading record which cannot be tampered, so as to improve the recognition precision and efficiency.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation, in particular to the field of vehicle identification number (VIN) reading technology, and specifically to a method and system for adaptively reading a vehicle identification number. Background Art

[0002] The Vehicle Identification Number (VIN) is a code used to uniquely identify vehicles worldwide. It usually consists of 17 characters and contains information about the vehicle manufacturer, model, year of production, engine type, etc. Accurate reading of the VIN code is of great significance to vehicle management, identity authentication, intelligent traffic management, autonomous driving and other fields. Currently, the method of reading the vehicle identification code mainly relies on manual entry, barcode scanning, fixed readers, RFID and other methods. However, these traditional methods have the following technical bottlenecks: 1. Low reading accuracy and efficiency: Traditional fixed readers often rely on strict environmental conditions, such as light intensity, distance, angle, etc., to read vehicle identification codes efficiently. Once these conditions change (such as insufficient light, inappropriate vehicle angle, etc.), the reading accuracy will be greatly reduced, resulting in recognition failure or reading errors, affecting overall work efficiency.

[0003] 2. Lack of adaptive capabilities: Traditional vehicle identification code reading methods usually work based on preset conditions and lack the ability to adapt to different environmental changes. For example, at night or in bad weather conditions, vehicle identification codes are often difficult to recognize, and existing technologies find it difficult to dynamically adjust image acquisition and processing parameters to adapt to changes in ambient lighting, vehicle angle or distance.

[0004] 3. More manual intervention: Although some advanced barcode scanning or RFID technologies have partially solved the automation problem, manual intervention or equipment adjustment is still required, especially in a changeable and complex environment, where manual intervention is frequent. Manual operation not only wastes time, but also increases the risk of human error.

[0005] 4. Lighting and viewing angle issues: The position of the vehicle identification code is usually affected by the vehicle shape, angle and obstacles. Existing image recognition systems often require precise angles and lighting conditions to read accurately. If there are obstructions or complex backgrounds in the recognition area, existing methods are difficult to effectively handle these interferences.

[0006] In order to solve these problems, the industry urgently needs a technology that can adaptively read vehicle identification codes in complex environments, adjust the image acquisition and recognition process in real time to improve recognition accuracy and efficiency, and achieve a high level of automation and low human intervention. Summary of the invention

[0007] In view of this, in order to solve the problems of insufficient accuracy, adaptability and automation in traditional vehicle identification code reading technology, the purpose of the present invention is to propose a method and system for adaptively reading vehicle identification codes, which combines deep learning, sensor data, adaptive adjustment and real-time data transmission to adaptively read vehicle identification codes, which can significantly improve the reading efficiency, accuracy and adaptability of vehicle identification codes, and is widely used in intelligent transportation, autonomous driving, on-board identification and other fields, greatly promoting the automation and intelligent development of intelligent transportation systems.

[0008] To achieve the above object, the present invention provides the following technical solutions: Based on the above objectives, in a first aspect, the present invention provides a method for adaptively reading a vehicle identification code, comprising the following steps: The installed sensors collect multi-modal sensing data of vehicles entering the target area in real time; Time synchronization and fusion of the collected multi-modal sensor data are performed to generate multi-dimensional fusion data with a unified data structure according to the collection time; The image data collected from multiple perspectives in the multi-dimensional fusion data are reconstructed in three dimensions through stereo matching to generate a three-dimensional model of the target area; and based on the fusion of the depth information, vehicle position and motion state when the vehicle enters the target area, the spatial position and shape of the vehicle in the three-dimensional model are constructed, and the area where the vehicle identification code is located is determined; According to the three-dimensional reconstructed stereo model and the spatial position and shape of the vehicle in the stereo model, the optimal viewing angle for reading the vehicle identification code is analyzed, and the camera angle and distance are adaptively adjusted to capture the target image containing the vehicle identification code area; The captured target image is enhanced using histogram equalization, and the vehicle identification code area in the target image is identified based on a deep convolutional neural network. Character recognition is performed on the extracted vehicle identification code area, and the successfully read vehicle identification code character information is encrypted and stored through the blockchain to generate an unalterable vehicle identification code reading record.

[0009] As a further solution of the present invention, the installed sensors include: an RGB camera for capturing image information of the vehicle and providing clear color images; an infrared thermal imaging camera for obtaining thermal imaging images under low light or night conditions and identifying thermal signals of the vehicle; a lidar for providing depth data and extracting the spatial position and shape of the vehicle; a sonar sensor for detecting the distance of the vehicle and measuring the distance to the target object; and an inertial measurement unit for providing motion state information of the vehicle.

[0010] As a further solution of the present invention, when the installed sensors collect multimodal sensor data of the vehicle entering the target area in real time, the sensors are deployed in the target area for real-time data collection, the image data is captured by the camera, the depth information is provided by the lidar, the vehicle acceleration and steering angle are measured by the inertial measurement unit, the sonar sensor calculates the distance between the vehicle and the object, and the multimodal sensor data is synchronized using timestamp synchronization, and multidimensional fusion data with a unified data structure is generated after fusion.

[0011] As a further solution of the present invention, generating a three-dimensional model of the target area includes the following steps: Collect image data of the target area through multiple viewing angle cameras and record the camera's internal and external reference information; Perform denoising and dedistortion processing on the collected image data to extract feature points in the image; Generate a disparity map of the target area through feature matching and disparity calculation; According to the disparity information and camera parameters, the three-dimensional coordinates of the target area are restored through triangulation; The three-dimensional point clouds from multiple perspectives are registered and fused to generate a three-dimensional model of the target area.

[0012] As a further solution of the present invention, multiple viewing angle cameras are RGB cameras and infrared thermal imaging cameras deployed in the target area in binocular, quad-view or circular arrays. When collecting image data of the target area, the timestamp, internal parameters (focal length, optical center, distortion coefficient) and external parameters (relative position and direction of the camera) of each camera are synchronously recorded. After processing the image data, the key points in the image are extracted using scale-invariant feature transformation, and matched according to scale and rotation, and matching point pairs are found in images of different viewing angles to form feature points.

[0013] As a further solution of the present invention, when the collected image data is dedistorted, the radial distortion coefficient or the tangential distortion coefficient obtained by camera calibration is used to perform image correction, wherein the radial distortion is calculated using the pixel coordinates in the distorted image and the radial distortion coefficient, and the calculation formula is: In the formula, is the pixel coordinate after dedistortion, is the pixel coordinate in the distorted image; , is the radial distortion coefficient obtained by camera calibration; Among them, the tangential distortion is calculated using the pixel coordinates in the distorted image and the tangential distortion coefficient, and the calculation formula is: In the formula, , is the tangential distortion coefficient obtained by camera calibration.

[0014] As a further solution of the present invention, when generating a disparity map of the target area through feature matching and disparity calculation, the disparity of the matching point pair is calculated to determine the difference in pixel position of the same scene point under different viewing angles, wherein the disparity of the point in the image is determined by the difference between the horizontal coordinate of the matching point in the first viewing angle image and the horizontal coordinate of the matching point in the second viewing angle image, and the three-dimensional coordinates of the scene are restored by triangulation using the intrinsic and extrinsic parameters of the camera and the calculated disparity information. The calculation formula for the depth of the three-dimensional point cloud is: In the formula, is the depth of the target point, that is, the distance between the target point and the camera; is the focal length of the camera, is the baseline of the camera, i.e., the distance between the two cameras; is the disparity between corresponding points.

[0015] As a further solution of the present invention, when registering and fusing the three-dimensional point clouds of multiple perspectives to generate a stereoscopic model of the target area, a three-dimensional point cloud is generated by converting the depth information of each matching point into three-dimensional coordinates to represent the three-dimensional structure of the target area; wherein the three-dimensional point cloud is: In the formula, , is the world coordinate of the 3D point; , is the pixel coordinate of the target point in the image; , is the pixel coordinate corresponding to the optical center of the camera intrinsic parameter; , The focal length of the camera is obtained by camera calibration; The depth of the target point is calculated by disparity; Generate 3D point clouds from each perspective for multi-view image data, and fuse the generated 3D point clouds into the same coordinate system through point cloud registration; Align point clouds from different perspectives, optimize relative positions, merge point clouds from different perspectives, remove duplicate points, and generate a three-dimensional model of the target area; The point cloud is converted into the three-dimensional surface of the stereo model through Delaunay triangulation, and a smooth three-dimensional surface is reconstructed on the stereo model.

[0016] As a further solution of the present invention, determining the area where the vehicle identification code is located includes the following steps: The depth information of each matching point is converted into three-dimensional coordinates, and a three-dimensional point cloud is generated from each perspective for multi-perspective image data; According to the depth map and the three-dimensional model, the spatial area corresponding to the vehicle identification code is determined through image processing, and the vehicle identification code is extracted.

[0017] As a further solution of the present invention, the best viewing angle for reading the vehicle identification code is analyzed based on the three-dimensionally reconstructed stereoscopic model and the spatial position and shape of the vehicle in the stereoscopic model, including: The geometric position of the vehicle identification code area is calculated based on the three-dimensional reconstructed stereo model, and the optimal viewing angle is determined by the intersection with the camera's visible area; Starting from the camera, calculate the camera's viewing cone range and determine whether the identification code area is within the viewing cone. If not, adjust the camera's angle and distance until the identification code area enters the field of view. The position and angle of the camera are adjusted by the gradient descent method, and based on the size of the identification code and the focal length of the camera, the focal length of the camera and the distance to the target are adjusted so that the identification code area is centered in the image; When the camera angle and distance are optimized, the camera is used to capture an image of the target area and extract the vehicle identification code. The vehicle identification code is detected and decoded on the captured image, and clear identification code information is extracted from the image.

[0018] As a further solution of the present invention, when the captured target image is enhanced by using histogram equalization, the image is transformed by a cumulative distribution function, the pixel distribution of the input image is equalized, and the gray value range of the image is transformed as follows when the histogram is equalized: In the formula, is the transformed grayscale value, indicating the grayscale level of the original image The new grayscale after equalization is mapped to, where the grayscale value range of the image is 0 to 255, and the grayscale histogram of the image is , Indicates the number of pixels of a certain gray level in the image. is the index of the image gray level, , represents each gray value of the image; is the number of gray levels in the image. For an 8-bit image, , the gray level ranges from 0 to 255; is the total number of pixels in the image; is the gray value in the original image The pixel frequency of For arrive The cumulative sum of the frequencies of all gray values ​​​​is less than or equal to The total number of all pixels; After histogram equalization transformation, the gray value of each pixel in the original image is mapped to a new balanced grayscale value .

[0019] As a further solution of the present invention, when identifying the vehicle identification code area in the target image based on the deep convolutional neural network, the convolution layer of the deep convolutional neural network is used to extract local features of the image, the convolution operation convolves the input image through the convolution kernel to extract the feature map, the pooling layer reduces the size of the feature map through downsampling, the fully connected layer flattens the features extracted by the convolution layer, and classifies them through the fully connected layer, outputs the identified target category and its position, uses the YOLO model to locate the license plate area in the image, and automatically detects the license plate area in the image.

[0020] As a further solution of the present invention, when character recognition is performed on the extracted vehicle identification code area, the extracted license plate area image is input into the character recognition model, and the recognized license plate number is transmitted to the encrypted evidence, and the encrypted evidence is stored through the blockchain to generate a tamper-proof vehicle identification code reading record; wherein, RSA public key encryption is used to encrypt the vehicle identification code information during encrypted evidence storage, and the encrypted vehicle identification code data will be stored in the blockchain, and each encrypted evidence record is a block, including a timestamp, an encrypted vehicle identification code, a block hash and a previous block hash, wherein the timestamp records the time when the vehicle identification code is read, and the encrypted vehicle identification code is the encrypted vehicle identification code information; the encrypted vehicle identification code is stored through blockchain technology to generate a tamper-proof record.

[0021] In a second aspect, the present invention provides a system for adaptively reading a vehicle identification code, comprising: Multimodal sensor group: including cameras, laser radar (LiDAR) and position sensors, responsible for real-time collection of multimodal sensor data of vehicles entering the target area; the camera collects image data, the laser radar collects depth data, and the position sensor obtains the vehicle's motion state and position; Data synchronization and fusion module: used to perform time synchronization and fusion of multimodal sensor data, align data from different sensors by timestamps, and fuse them into a unified multidimensional data structure.

[0022] 3D reconstruction module: used to perform stereo matching on image data from multiple perspectives to generate a 3D model of the target area. Through stereo matching, the depth information of the target area is constructed according to the perspective differences of different sensors and cameras to generate an accurate 3D model. Vehicle shape positioning module: used to infer the specific spatial position and shape of the vehicle in the three-dimensional model based on the vehicle's depth information, vehicle position, and motion status data, analyze the dynamic changes of the vehicle in the target area, determine the vehicle's spatial position, and accurately locate the vehicle identification code area; Optimal viewing angle adjustment module: used to calculate the optimal viewing angle based on the 3D model and the vehicle position and shape information, and automatically adjust the angle and distance of the camera to capture the image area containing the vehicle identification code; Vehicle identification number extraction module: enhances the captured image and uses a deep convolutional neural network to extract the vehicle identification number area; Character recognition module: performs character segmentation and recognition on the extracted vehicle identification code area to generate character information of the vehicle identification code; Blockchain evidence storage module: The successfully read vehicle identification code information is encrypted and stored to generate an unalterable vehicle identification code reading record.

[0023] The system for adaptively reading vehicle identification codes of the present invention automatically optimizes the angle of the camera to improve the image quality through data collection, data fusion, three-dimensional reconstruction, and optimal viewing angle calculation of multimodal sensors, and uses a deep learning model to perform license plate character recognition, and finally ensures the security and non-tamperability of the data through blockchain evidence storage technology. This system not only improves the accuracy of vehicle identification code reading, but also processes and stores information in real time, ensuring the credibility and security of the license plate recognition process.

[0024] In another aspect of the present invention, a computer device is provided, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, any one of the above methods for adaptively reading a vehicle identification code according to the present invention is executed.

[0025] In another aspect of the present invention, a computer-readable storage medium is provided, which stores computer program instructions. When the computer program instructions are executed, any one of the above methods for adaptively reading a vehicle identification code according to the present invention is implemented.

[0026] Compared with the prior art, the method and system for adaptively reading vehicle identification codes proposed by the present invention have the following beneficial effects: 1. The present invention uses real-time data acquisition (including images, depth, location and other information) from multi-modal sensors, and the system can fully and accurately perceive the vehicle status in the target area. This data fusion and stereo reconstruction technology enables the system to effectively eliminate image blur and misidentification caused by environmental factors (such as light changes, occlusion, etc.) or sensor errors, thereby greatly improving the accuracy of vehicle identification code reading.

[0027] 2. By calculating the optimal viewing angle, the system of the present invention can avoid the problem of license plate obstruction or blurring caused by different vehicle positions, thereby ensuring that the vehicle identification code can be clearly read in various scenarios; based on the real-time calculation results, the angle and distance of the camera are automatically adjusted so that the shooting angle of the vehicle identification code always remains in the optimal state and adapts to different vehicle sizes, shapes or motion states. The present invention dynamically calculates and adjusts the angle and distance of the camera according to the spatial position and shape of the vehicle in the target area to ensure that the best recognition image is obtained. This adaptive adjustment function can be flexibly adjusted according to actual needs in different situations (such as changes in vehicle speed, changes in viewing angle, etc.), reducing the need for human intervention or setting fixed parameters, and optimizing the effect of image acquisition.

[0028] 3. The present invention uses image enhancement technology (histogram equalization) to process the captured target image, which can significantly improve the contrast and clarity of the image and enhance the recognizability of the vehicle identification code area. The image enhancement function can effectively solve the problem of poor image quality caused by camera quality, environmental interference or shooting angle problems, and ensure that the subsequent recognition algorithm can accurately read the vehicle identification code. By encrypting the successfully read vehicle identification code information and storing it in the blockchain, the security, immutability and transparency of the reading record can be ensured. This evidence storage process provides an unchangeable historical record, greatly enhances the legal effect and credibility of the data, and prevents tampering or forgery of the vehicle identification code information.

[0029] In summary, the method and system for adaptively reading vehicle identification codes of the present invention combine deep learning, sensor data, adaptive adjustment and real-time data transmission to adaptively read vehicle identification codes, which can significantly improve the reading efficiency, accuracy and adaptability of vehicle identification codes, and are widely used in intelligent transportation, autonomous driving, on-board identification and other fields, greatly promoting the automation and intelligent development of intelligent transportation systems.

[0030] These and other aspects of the present application will be more concise and understandable in the following description of the embodiments. It should be understood that the above general description and the following detailed description are only exemplary and explanatory and cannot limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the following briefly introduces the drawings required for use in the exemplary embodiments or related technical descriptions. The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the drawings: Figure 1 The present invention is a flowchart of a method for adaptively reading a vehicle identification code according to an embodiment of the present invention.

[0032] Figure 2 The present invention is a flowchart of generating a three-dimensional model of a target area in a method for adaptively reading a vehicle identification code according to an embodiment of the present invention.

[0033] Figure 3 The present invention is a flowchart of analyzing the best viewing angle for reading a vehicle identification code in a method for adaptively reading a vehicle identification code according to an embodiment of the present invention. DETAILED DESCRIPTION

[0034] Below, the present application is further described in conjunction with the accompanying drawings and specific implementation methods. It should be noted that, under the premise of no conflict, the various embodiments or technical features described below can be arbitrarily combined to form a new embodiment.

[0035] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the embodiments of the present invention are further described in detail below in combination with specific embodiments and with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0036] It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are intended to distinguish two non-identical entities or non-identical parameters with the same name. It can be seen that "first" and "second" are only for the convenience of expression and should not be understood as limitations on the embodiments of the present invention. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, other steps or units inherent to a process, method, system, product or device that includes a series of steps or units.

[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0038] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0039] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0040] Aiming at the problems of insufficient accuracy, adaptability and automation in traditional vehicle identification code reading technology, the present invention proposes a method and system for adaptively reading vehicle identification codes, which combines deep learning, sensor data, adaptive adjustment and real-time data transmission to adaptively read vehicle identification codes, and can significantly improve the reading efficiency, accuracy and adaptability of vehicle identification codes, and is widely used in intelligent transportation, automatic driving, vehicle-mounted identification and other fields, greatly promoting the automation and intelligent development of intelligent transportation systems.

[0041] See also Figure 1 As shown, an embodiment of the present invention provides a method for adaptively reading a vehicle identification code, the method comprising the following steps: Step S10: collect multimodal sensing data of the vehicle entering the target area in real time through the installed sensors.

[0042] In this step, the installed sensors include: an RGB camera for capturing image information of the vehicle and providing clear color images; an infrared thermal imaging camera for obtaining thermal imaging images in low light or night conditions and identifying the thermal signature of the vehicle; a lidar for providing depth data and extracting the spatial position and shape of the vehicle; a sonar sensor for detecting the distance of the vehicle and measuring the distance to the target object; and an inertial measurement unit for providing information on the vehicle's motion status.

[0043] Step S20: time-synchronize and fuse the collected multimodal sensor data, and generate multidimensional fused data with a unified data structure according to the collection time.

[0044] In this step, when the installed sensors collect multimodal sensor data of the vehicle entering the target area in real time, the sensors are deployed in the target area for real-time data collection, the camera captures image data, the lidar provides depth information, the inertial measurement unit measures the vehicle acceleration and steering angle, the sonar sensor calculates the distance between the vehicle and the object, and the timestamp synchronization is used to synchronize the multimodal sensor data, and after fusion, multi-dimensional fusion data with a unified data structure is generated.

[0045] The present invention uses real-time data acquisition (including images, depth, location and other information) from multi-modal sensors to enable the system to fully and accurately perceive the status of vehicles in the target area. This data fusion and stereo reconstruction technology enables the system to effectively eliminate image blur and misidentification caused by environmental factors (such as light changes, occlusion, etc.) or sensor errors, thereby greatly improving the accuracy of vehicle identification code reading.

[0046] Step S30, the image data collected from multiple perspectives in the multi-dimensional fusion data are three-dimensionally reconstructed through stereo matching to generate a three-dimensional model of the target area; and based on the fusion of the depth information, vehicle position and motion state of the vehicle entering the target area, the spatial position and shape of the vehicle in the three-dimensional model are constructed, and the area where the vehicle identification code is located is determined.

[0047] In this step, see Figure 2 As shown, generating a stereo model of the target area includes the following steps: Step S301: collect image data of a target area through multiple viewing angle cameras, and record the camera's intrinsic and extrinsic parameter information.

[0048] In this embodiment, the multiple viewing angle cameras are RGB cameras and infrared thermal imaging cameras deployed in the target area in binocular, quadruple or circular array. Among them, the binocular configuration is that two cameras are arranged opposite to each other to capture different viewing angles of the same area; the quadruple configuration is that four cameras cover the target area from multiple angles; the circular array is that multiple cameras are distributed in a circle around the target area, which can capture the image data of the target area in all directions. When collecting images, each camera synchronously records the following information: Timestamp: Ensures data from each camera is synchronized.

[0049] Internal parameter information: including focal length, optical center, distortion coefficient, etc.

[0050] External parameter information: including the relative position and orientation of the cameras (for example, the relative posture between cameras).

[0051] Step S302: performing denoising and dedistortion processing on the collected image data to extract feature points in the image; Step S303, generating a disparity map of the target area through feature matching and disparity calculation; Step S304: restore the three-dimensional coordinates of the target area through triangulation according to the disparity information and camera parameters; Step S305: register and fuse the three-dimensional point clouds from multiple perspectives to generate a three-dimensional model of the target area.

[0052] When collecting image data of the target area, the timestamp, intrinsic parameters (focal length, optical center, distortion coefficient) and extrinsic parameters (relative position and direction of the camera) of each camera are recorded synchronously. After processing the image data, the key points in the image are extracted using scale-invariant feature transformation, and matched according to scale and rotation. Matching point pairs are found in images of different perspectives to form feature points.

[0053] In step S302, when the collected image data is subjected to dedistortion processing, the radial distortion coefficient or the tangential distortion coefficient obtained by camera calibration is used to perform image correction, wherein the radial distortion is calculated using the pixel coordinates in the distorted image and the radial distortion coefficient, and the calculation formula is: In the formula, is the pixel coordinate after dedistortion, is the pixel coordinate in the distorted image; , is the radial distortion coefficient obtained by camera calibration; Among them, the tangential distortion is calculated using the pixel coordinates in the distorted image and the tangential distortion coefficient, and the calculation formula is: In the formula, , is the tangential distortion coefficient obtained by camera calibration.

[0054] In step S303, when generating a disparity map of the target area through feature matching and disparity calculation, the disparity of the matching point pairs is calculated to determine the pixel position difference of the same scene point under different viewing angles, wherein the disparity of the point in the image is determined by the difference between the horizontal coordinate of the matching point in the first viewing angle image and the horizontal coordinate of the matching point in the second viewing angle image, and the three-dimensional coordinates of the scene are restored by triangulation using the intrinsic and extrinsic parameters of the camera and the calculated disparity information. The calculation formula for the depth of the three-dimensional point cloud is: In the formula, is the depth of the target point, that is, the distance between the target point and the camera; is the focal length of the camera, is the baseline of the camera, i.e., the distance between the two cameras; is the disparity between corresponding points.

[0055] In step S305, the three-dimensional point clouds of multiple perspectives are registered and fused to generate a stereoscopic model of the target area. The depth information of each matching point is converted into three-dimensional coordinates to generate a three-dimensional point cloud for representing the three-dimensional structure of the target area. The three-dimensional point cloud is: In the formula, , is the world coordinate of the 3D point; , is the pixel coordinate of the target point in the image; , is the pixel coordinate corresponding to the optical center of the camera intrinsic parameter; , The focal length of the camera is obtained by camera calibration; The depth of the target point is calculated by disparity; Generate 3D point clouds from each perspective for multi-view image data, and fuse the generated 3D point clouds into the same coordinate system through point cloud registration; Align point clouds from different perspectives, optimize relative positions, merge point clouds from different perspectives, remove duplicate points, and generate a three-dimensional model of the target area; The point cloud is converted into the three-dimensional surface of the stereo model through Delaunay triangulation, and a smooth three-dimensional surface is reconstructed on the stereo model.

[0056] In this embodiment, determining the area where the vehicle identification code is located includes the following steps: The depth information of each matching point is converted into three-dimensional coordinates, and a three-dimensional point cloud is generated from each perspective for multi-perspective image data; According to the depth map and the three-dimensional model, the spatial area corresponding to the vehicle identification code is determined through image processing, and the vehicle identification code is extracted.

[0057] Step S40: Analyze the best viewing angle for reading the vehicle identification code based on the three-dimensionally reconstructed stereo model and the spatial position and shape of the vehicle in the stereo model, and adaptively adjust the camera angle and distance to capture a target image containing the vehicle identification code area.

[0058] In this step, see Figure 3 As shown, according to the three-dimensional reconstructed stereo model and the spatial position and shape of the vehicle in the stereo model, the best viewing angle for reading the vehicle identification code is analyzed, including: Step S401: Calculate the geometric position of the vehicle identification code area based on the three-dimensional reconstructed stereo model, and determine the best viewing angle through the intersection with the camera visible area.

[0059] Among them, during 3D reconstruction, the image data collected by the multi-view camera is used for 3D reconstruction to generate a 3D model of the target area; in the positioning identification code area, the geometric position of the vehicle identification code area is calculated based on the spatial position and shape of the vehicle, combined with the actual size, posture and area where the identification code is located. Assume that the position coordinates of the vehicle obtained by the 3D reconstruction algorithm are (X, Y, Z), the vehicle identification code is located on a plane in front of the vehicle, and the coordinates on the plane are ( , , ).

[0060] Step S402: Starting from the camera, calculate the viewing cone range of the camera, and determine whether the identification code area is within the viewing cone. If not, adjust the angle and distance of the camera until the identification code area enters the field of view.

[0061] When calculating the view cone, the view cone of the camera is determined by the field of view (FOV), focal length, and camera position. The view cone range of the camera is calculated based on the camera's internal and external parameters. For example, the focal length of the camera is , the horizontal field of view of the camera is , the vertical field of view is , defines the range of the viewing cone. If the identification code area ( , , ) is not within the viewing frustum, adjust the position and angle of the camera.

[0062] Step S403: adjust the position and angle of the camera by using the gradient descent method, and adjust the focal length of the camera and the distance to the target based on the size of the identification code and the focal length of the camera, so that the identification code area is centered in the image.

[0063] If the identification code area is small, the area can be enlarged by reducing the focal length or getting closer to the target area to ensure its clarity.

[0064] Step S404: After the camera angle and distance are optimized, the camera is used to capture an image of the target area, and the vehicle identification code is extracted. The vehicle identification code is detected and decoded on the captured image, and clear identification code information is extracted from the image.

[0065] For example, the image captured by the camera after the final adjustment clearly shows the license plate (identification code area) of the vehicle, and the image now includes the license plate number, such as "XYZ1234".

[0066] By calculating the optimal viewing angle, the system of the present invention can avoid the problem of license plate obstruction or blurring caused by different vehicle positions, thereby ensuring that the vehicle identification code can be clearly read in various scenarios; based on the real-time calculation results, the angle and distance of the camera are automatically adjusted so that the shooting angle of the vehicle identification code always remains in the optimal state and adapts to different vehicle sizes, shapes or motion states. The present invention dynamically calculates and adjusts the angle and distance of the camera according to the spatial position and shape of the vehicle in the target area to ensure that the best recognition image is obtained. This adaptive adjustment function can be flexibly adjusted according to actual needs in different situations (such as changes in vehicle speed, changes in viewing angle, etc.), reducing the need for human intervention or setting fixed parameters, and optimizing the effect of image acquisition.

[0067] Step S50: The captured target image is enhanced by histogram equalization, and the vehicle identification code area in the target image is identified based on a deep convolutional neural network. Character recognition is performed on the extracted vehicle identification code area, and the successfully read vehicle identification code character information is encrypted and stored through the blockchain to generate an unalterable vehicle identification code reading record.

[0068] In this step, when the captured target image is enhanced by histogram equalization, the image is transformed by cumulative distribution function, the pixel distribution of the input image is equalized, and the gray value range of the image is transformed as follows when the histogram is equalized: In the formula, is the transformed grayscale value, indicating the grayscale level of the original image The new grayscale after equalization is mapped to, where the grayscale value range of the image is 0 to 255, and the grayscale histogram of the image is , Indicates the number of pixels of a certain gray level in the image. is the index of the image gray level, , represents each gray value of the image; is the number of gray levels in the image. For an 8-bit image, , the gray level ranges from 0 to 255; is the total number of pixels in the image; is the gray value in the original image The pixel frequency; For arrive The cumulative sum of the frequencies of all gray values ​​​​is less than or equal to The total number of all pixels; After histogram equalization transformation, the gray value of each pixel in the original image is mapped to a new balanced grayscale value .

[0069] In this embodiment, when the vehicle identification code area in the target image is identified based on the deep convolutional neural network, the convolution layer of the deep convolutional neural network is used to extract local features of the image. The convolution operation convolves the input image through the convolution kernel to extract the feature map. The pooling layer reduces the size of the feature map by downsampling. The fully connected layer flattens the features extracted by the convolution layer and classifies them through the fully connected layer. The identified target category and its position are output, and the YOLO model is used to locate the license plate area in the image, and the license plate area in the image is automatically detected.

[0070] In this embodiment, when character recognition is performed on the extracted vehicle identification code area, the extracted license plate area image is input into the character recognition model, and the recognized license plate number is passed to the encrypted evidence, and the encrypted evidence is stored through the blockchain to generate a tamper-proof vehicle identification code reading record; wherein, RSA public key encryption is used to encrypt the vehicle identification code information during encrypted evidence storage, and the encrypted vehicle identification code data will be stored in the blockchain, and each encrypted evidence record is a block, including a timestamp, an encrypted vehicle identification code, a block hash and a previous block hash, wherein the timestamp records the time when the vehicle identification code is read, and the encrypted vehicle identification code is the encrypted vehicle identification code information; the encrypted vehicle identification code is stored through blockchain technology to generate a tamper-proof record.

[0071] The present invention uses image enhancement technology (histogram equalization) to process the captured target image, which can significantly improve the contrast and clarity of the image and enhance the recognizability of the vehicle identification code area. The image enhancement function can effectively solve the problem of poor image quality caused by camera quality, environmental interference or shooting angle problems, and ensure that the subsequent recognition algorithm can accurately read the vehicle identification code. By encrypting the successfully read vehicle identification code information and storing it in the blockchain, the security, immutability and transparency of the reading record can be ensured. This evidence storage process provides an unchangeable historical record, greatly enhances the legal effect and credibility of the data, and prevents tampering or forgery of the vehicle identification code information.

[0072] To sum up, the method of adaptively reading vehicle identification codes of the present invention, which combines deep learning, sensor data, adaptive adjustment and real-time data transmission to adaptively read vehicle identification codes, can significantly improve the reading efficiency, accuracy and adaptability of vehicle identification codes, and is widely used in intelligent transportation, autonomous driving, on-board identification and other fields, greatly promoting the automation and intelligent development of intelligent transportation systems.

[0073] It should be noted that the above-mentioned figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0074] It should be understood that, although described in a certain order, these steps are not necessarily performed in sequence in the above order. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, a part of the steps of the present embodiment may include a plurality of steps or a plurality of stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a part of the steps or stages in other steps or other steps.

[0075] According to a second aspect of the embodiments of the present invention, the present invention further provides a system for adaptively reading a vehicle identification code, comprising: Multimodal sensor group: including cameras, laser radar (LiDAR) and position sensors, responsible for real-time collection of multimodal sensor data of vehicles entering the target area; the camera collects image data, the laser radar collects depth data, and the position sensor obtains the vehicle's motion state and position; Data synchronization and fusion module: used to synchronize and fuse multimodal sensor data, align data from different sensors by timestamps, and fuse them into a unified multidimensional data structure; 3D reconstruction module: used to perform stereo matching on image data from multiple perspectives to generate a 3D model of the target area. Through stereo matching, the depth information of the target area is constructed according to the perspective differences of different sensors and cameras to generate an accurate 3D model. Vehicle shape positioning module: used to infer the specific spatial position and shape of the vehicle in the three-dimensional model based on the vehicle's depth information, vehicle position, and motion status data, analyze the dynamic changes of the vehicle in the target area, determine the vehicle's spatial position, and accurately locate the vehicle identification code area; Optimal viewing angle adjustment module: used to calculate the optimal viewing angle based on the 3D model and the vehicle position and shape information, and automatically adjust the angle and distance of the camera to capture the image area containing the vehicle identification code; Vehicle identification number extraction module: enhances the captured image and uses a deep convolutional neural network to extract the vehicle identification number area; Character recognition module: performs character segmentation and recognition on the extracted vehicle identification code area to generate character information of the vehicle identification code; Blockchain evidence storage module: The successfully read vehicle identification code information is encrypted and stored to generate an unalterable vehicle identification code reading record.

[0076] The system for adaptively reading vehicle identification codes of the present invention automatically optimizes the angle of the camera to improve the image quality through data collection, data fusion, three-dimensional reconstruction, and optimal viewing angle calculation of multimodal sensors, and uses a deep learning model to perform license plate character recognition, and finally ensures the security and non-tamperability of the data through blockchain evidence storage technology. This system not only improves the accuracy of vehicle identification code reading, but also processes and stores information in real time, ensuring the credibility and security of the license plate recognition process.

[0077] Through the above detailed steps, the system for adaptively reading vehicle identification codes of the present invention is used to execute the steps of the method for adaptively reading vehicle identification codes in the above embodiment, which will not be described in detail here.

[0078] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope disclosed in the embodiments of the present invention as defined in the claims. The functions, steps and / or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any particular order. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual form, they may also be understood as multiple unless explicitly limited to the singular.

[0079] A person skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the disclosure of the embodiments of the present invention (including the claims) is limited to these examples; under the concept of the embodiments of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the above embodiments of the present invention, which are not provided in detail for the sake of simplicity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the embodiments of the present invention.

Claims

1. A method for adaptively reading a vehicle identification code, characterized in that: The method comprises the following steps: The installed sensors collect multi-modal sensing data of vehicles entering the target area in real time; Time synchronization and fusion of the collected multi-modal sensor data are performed to generate multi-dimensional fusion data with a unified data structure according to the collection time; The image data collected from multiple perspectives in the multi-dimensional fusion data are reconstructed in three dimensions through stereo matching to generate a three-dimensional model of the target area; and based on the fusion of the depth information, vehicle position and motion state when the vehicle enters the target area, the spatial position and shape of the vehicle in the three-dimensional model are constructed, and the area where the vehicle identification code is located is determined; According to the three-dimensional reconstructed stereo model and the spatial position and shape of the vehicle in the stereo model, the optimal viewing angle for reading the vehicle identification code is analyzed, and the camera angle and distance are adaptively adjusted to capture the target image containing the vehicle identification code area; The captured target image is enhanced using histogram equalization, and the vehicle identification code area in the target image is identified based on a deep convolutional neural network. Character recognition is performed on the extracted vehicle identification code area, and the successfully read vehicle identification code character information is encrypted and stored through the blockchain to generate an unalterable vehicle identification code reading record.

2. The method for adaptively reading a vehicle identification code according to claim 1, characterized in that: The installed sensors include: RGB cameras for capturing image information of the vehicle and providing clear color images; infrared thermal imaging cameras for obtaining thermal imaging images in low-light or night conditions and identifying the thermal signature of the vehicle; lidars for providing depth data and extracting the spatial position and shape of the vehicle; sonar sensors for detecting the distance of the vehicle and measuring the distance to the target object; and inertial measurement units for providing information on the vehicle's motion status.

3. The method for adaptively reading a vehicle identification code according to claim 2, characterized in that: When the installed sensors collect multimodal sensor data of the vehicle entering the target area in real time, the sensors are deployed in the target area for real-time data collection, the camera captures image data, the lidar provides depth information, the inertial measurement unit measures the vehicle acceleration and steering angle, the sonar sensor calculates the distance between the vehicle and the object, and the timestamp synchronization is used to synchronize the multimodal sensor data, and after fusion, multi-dimensional fusion data with a unified data structure is generated.

4. The method for adaptively reading a vehicle identification code according to claim 3, characterized in that: Generating a stereo model of the target area includes the following steps: Collect image data of the target area through multiple viewing angle cameras and record the camera's internal and external reference information; Perform denoising and dedistortion processing on the collected image data to extract feature points in the image; Generate a disparity map of the target area through feature matching and disparity calculation; According to the disparity information and camera parameters, the three-dimensional coordinates of the target area are restored through triangulation; The three-dimensional point clouds from multiple perspectives are registered and fused to generate a three-dimensional model of the target area.

5. The method for adaptively reading a vehicle identification code according to claim 4, characterized in that: The multiple-view cameras are RGB cameras and infrared thermal imaging cameras deployed in the target area in binocular, quad-view or circular arrays. When collecting image data of the target area, the timestamp, intrinsic parameter and extrinsic parameter information of each camera are synchronously recorded. After processing the image data, the key points in the image are extracted using scale-invariant feature transformation, and matched according to scale and rotation. Matching point pairs are found in images of different viewpoints to form feature points.

6. The method for adaptively reading a vehicle identification code according to claim 5, characterized in that: When dedistorting the collected image data, the radial distortion coefficient or tangential distortion coefficient obtained by camera calibration is used for image correction. The radial distortion is calculated using the pixel coordinates in the distorted image and the radial distortion coefficient. The calculation formula is: In the formula, is the pixel coordinate after dedistortion, is the pixel coordinate in the distorted image; , is the radial distortion coefficient obtained by camera calibration; Among them, the tangential distortion is calculated using the pixel coordinates in the distorted image and the tangential distortion coefficient, and the calculation formula is: In the formula, , is the tangential distortion coefficient obtained by camera calibration.

7. The method for adaptively reading a vehicle identification code according to claim 6, characterized in that: Through feature matching and disparity calculation, when generating the disparity map of the target area, the disparity of the matching point pair is calculated to determine the pixel position difference of the same scene point under different viewing angles. The disparity of the point in the image is determined by the difference between the horizontal coordinate of the matching point in the first viewing angle image and the horizontal coordinate of the matching point in the second viewing angle image. The camera's intrinsic and extrinsic parameters, as well as the calculated disparity information, are used to restore the three-dimensional coordinates of the scene through triangulation. The calculation formula for the depth of the three-dimensional point cloud is: In the formula, is the depth of the target point, that is, the distance between the target point and the camera; is the focal length of the camera, is the baseline of the camera, i.e., the distance between the two cameras; is the disparity between corresponding points.

8. The method for adaptively reading a vehicle identification code according to claim 7, characterized in that: When registering and fusing 3D point clouds from multiple perspectives to generate a stereoscopic model of the target area, a 3D point cloud is generated by converting the depth information of each matching point into 3D coordinates to represent the 3D structure of the target area; the 3D point cloud is: In the formula, , is the world coordinate of the 3D point; , is the pixel coordinate of the target point in the image; , is the pixel coordinate corresponding to the optical center of the camera intrinsic parameter; , The focal length of the camera is obtained by camera calibration; The depth of the target point is calculated by disparity; Generate 3D point clouds from each perspective for multi-view image data, and fuse the generated 3D point clouds into the same coordinate system through point cloud registration; Align point clouds from different perspectives, optimize relative positions, merge point clouds from different perspectives, remove duplicate points, and generate a three-dimensional model of the target area; The point cloud is converted into the three-dimensional surface of the stereo model through Delaunay triangulation, and a smooth three-dimensional surface is reconstructed on the stereo model.

9. The method for adaptively reading a vehicle identification code according to claim 1, characterized in that: According to the 3D reconstructed stereo model and the spatial position and shape of the vehicle in the stereo model, the best viewing angle for reading the vehicle identification code is analyzed, including: The geometric position of the vehicle identification code area is calculated based on the three-dimensional reconstructed stereo model, and the optimal viewing angle is determined by the intersection with the camera's visible area; Starting from the camera, calculate the camera's viewing cone range and determine whether the identification code area is within the viewing cone. If not, adjust the camera's angle and distance until the identification code area enters the field of view. The position and angle of the camera are adjusted by the gradient descent method, and based on the size of the identification code and the focal length of the camera, the focal length of the camera and the distance to the target are adjusted so that the identification code area is centered in the image; When the camera angle and distance are optimized, the camera is used to capture an image of the target area and extract the vehicle identification code. The vehicle identification code is detected and decoded on the captured image, and clear identification code information is extracted from the image.

10. A system for adaptively reading vehicle identification codes, characterized in that: The method for adaptively reading a vehicle identification code according to any one of claims 1 to 9 comprises: A multimodal sensor group, used to collect multimodal sensor data of vehicles entering the target area in real time; The data synchronization and fusion module is used to synchronize and fuse multimodal sensor data, align the data from different sensors by timestamps, and fuse them into a unified multidimensional data structure; A 3D reconstruction module is used to perform stereo matching on image data from multiple perspectives to generate a 3D model of the target area; The vehicle shape positioning module is used to infer the specific spatial position and shape of the vehicle in the three-dimensional model based on the vehicle's depth information, vehicle position, and motion status data, analyze the dynamic changes of the vehicle in the target area, determine the vehicle's spatial position, and accurately locate the vehicle identification code area; The optimal viewing angle adjustment module is used to calculate the optimal viewing angle based on the three-dimensional model and the vehicle position and shape information, and automatically adjust the angle and distance of the camera to capture the image area containing the vehicle identification code; Vehicle identification number extraction module: enhances the captured image and uses a deep convolutional neural network to extract the vehicle identification number area; Character recognition module: performs character segmentation and recognition on the extracted vehicle identification code area to generate character information of the vehicle identification code; Blockchain evidence storage module: The successfully read vehicle identification code information is encrypted and stored to generate an unalterable vehicle identification code reading record.

Images