Roadside robot data processing method and system based on deep learning

Abstract

The invention provides a roadside robot data processing method and system based on deep learning. The method comprises the following steps: acquiring roadside environment information and a dynamic picture sequence of a vehicle; identifying static features and dynamic features of each target vehicle in the dynamic picture sequence based on a trained multi-target tracking model; matching pictures corresponding to each target state from the dynamic picture sequence to obtain a target picture set of each target vehicle in each target state; using the trained convolutional neural network model to identify dynamic data of a target vehicle bounding box, and determining a target license plate bounding box based on the dynamic data of the target vehicle bounding box; and performing optical character recognition on the target license plate bounding box by adopting an OCR recognition model to obtain license plate information of the target vehicle, and generating roadside management information of the target vehicle. According to the technical scheme provided by the invention, different roadside traffic condition requirements can be met, and meanwhile, the data processing efficiency is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and specifically to a roadside robot data processing method and system based on deep learning. Background Art

[0002] In recent years, with the continuous advancement of intelligent city construction, smart transportation, as an important part of it, has achieved unprecedented development. As an emerging application in the field of smart traffic management, roadside robots are gradually becoming popular in the streets and alleys of cities. Their increasingly widespread application has brought many conveniences to urban traffic management.

[0003] Most of the existing roadside robots are based on traditional algorithms and collect relevant information about vehicles through various sensors to achieve parking management. For example, cameras are used to collect images of vehicle appearance to identify license plate numbers, and distance sensors are used to determine the parking location of vehicles.

[0004] However, the inventors found that roadside robots based on traditional algorithms exposed a series of problems in actual application, such as certain limitations in data processing efficiency and scene adaptability. Faced with a large number of rapidly updated vehicle conditions, the traditional algorithm has complex operation logic and low efficiency when processing multiple dynamic targets. It is difficult for roadside robots to meet the needs of real-time detection and tracking of multiple vehicles. The urban environment is complex and changeable. Different road sections, time periods and weather conditions will affect the data collection and processing of roadside robots. In the process of license plate recognition, roadside robots based on traditional algorithms have difficulty in quickly and accurately identifying vehicle information in situations such as strong backlight or dark light environments, or when the license plate is blocked by snow, dust, shade, snow, and dust, which limits the adaptability of roadside robots.

[0005] In view of this, there is an urgent need for a roadside robot data processing method and system based on deep learning to improve the data processing efficiency and scene adaptability of the roadside robot. Summary of the invention

[0006] In view of this, the present application provides a roadside robot data processing method, system and storage medium based on deep learning to solve the technical problems in the prior art that the roadside robot data processing efficiency based on traditional algorithms is relatively low and the scene adaptability is weak.

[0007] In a first aspect, the present application provides a roadside robot data processing method based on deep learning, which is applied to a roadside robot, and the method comprises:

[0008] Obtain roadside environment information and dynamic image sequences of vehicles;

[0009] Identify static features and dynamic features of each target vehicle in the dynamic picture sequence based on the trained multi-target tracking model;

[0010] According to the static features and dynamic features of each target vehicle, images corresponding to each target state are matched from the dynamic image sequence to obtain a target image set of each target vehicle in each target state;

[0011] According to the target image set, a trained convolutional neural network model is used to identify dynamic data of a target vehicle bounding box, and based on the dynamic data of the target vehicle bounding box, a target license plate bounding box is determined;

[0012] The target license plate boundary frame is subjected to optical character recognition using an OCR recognition model to obtain the license plate information of the target vehicle, and the roadside management information of the target vehicle is generated by combining the dynamic data of the target vehicle boundary frame and the roadside environment information.

[0013] Preferably, the roadside robot data processing method based on deep learning further includes:

[0014] When the target license plate bounding box cannot be determined from the dynamic data of the target vehicle bounding box, based on the current motion state of the robot, according to the dynamic data of the target vehicle bounding box, and in combination with the roadside environment information, the robot motion path and obstacle avoidance data for reaching the target vehicle are generated;

[0015] When the robot moves to the target vehicle based on the motion path and obstacle avoidance data, an auxiliary image of the target vehicle is obtained; wherein the target vehicle boundary box corresponding to the auxiliary image includes a license plate boundary box.

[0016] Preferably, the trained multi-target tracking model identifies the static features and dynamic features of each target vehicle in the dynamic picture sequence, including:

[0017] Based on the trained multi-target tracking model, the data association algorithm is used to match and track the target vehicles in different frame images, and the static and dynamic features of each target vehicle are identified;

[0018] The static features include appearance color, category, and vehicle shape;

[0019] The dynamic features include the target vehicle's driving position coordinates, driving speed, and driving direction.

[0020] Preferably, the matching of pictures corresponding to each target state from the dynamic picture sequence according to the static features and dynamic features of each target vehicle to obtain a target picture set of each target vehicle in each target state includes:

[0021] A feature matching algorithm is used to process each frame of the dynamic picture sequence. Based on the static feature matching of the target vehicle in the image, the dynamic features of the target vehicle are compared with the dynamic features of the corresponding target state to calculate the matching degree.

[0022] When the matching degree reaches a preset threshold, the frame image is added as a target image to the target image set corresponding to the target vehicle.

[0023] Preferably, the architecture of the trained convolutional neural network model includes multiple convolutional layers, pooling layers and fully connected layers;

[0024] Among them, the convolution layer uses convolution kernels of different sizes to extract the features of different states of the target vehicle;

[0025] The pooling layer uses a global average pooling method to reduce the dimension of the feature map output by the convolution layer and retain important features;

[0026] The fully connected layer uses a feature fusion network to fuse the features output by the pooling layer, and through the multi-target recognition layer, performs bounding box prediction and recognition on multiple target vehicles while predicting and recognizing their state categories;

[0027] The data set used to train the convolutional neural network model contains a large number of target vehicle images in different scenes, different angles, and different lighting conditions, and is annotated with precise vehicle bounding box position information;

[0028] During the training process, stochastic gradient descent and its variants are used, and the cross entropy loss function is combined with the regression loss function to optimize the model parameters.

[0029] Preferably, each frame of the target image set is processed independently, and the position coordinates, width information, and height information of the target vehicle boundary box in the frame of the image are output to form a dynamic data sequence of the target vehicle boundary box;

[0030] Based on the width information in the dynamic data of the target vehicle bounding box, a target frame image corresponding to the target width information is determined, and based on the trained convolutional neural network model, a target license plate bounding box in the target frame image is identified.

[0031] Preferably, the image within the license plate boundary frame is subjected to size normalization processing, and the size normalized image is subjected to grayscale conversion, noise reduction, binarization, and tilt correction preprocessing in sequence;

[0032] Extract local features of characters in preprocessed images, capture contextual relationships between characters, and obtain global feature information through self-attention mechanism;

[0033] When there are multiple candidate license plate information in the recognition result, they are screened according to the preset confidence threshold. When the confidence is higher than the threshold, the result with the highest confidence is selected as the final license plate information;

[0034] The generated roadside management information of the target vehicle includes: license plate information of the target vehicle, entry information, parking coordinate information, and exit information of the target vehicle.

[0035] In a second aspect, the present application provides a roadside robot data processing system based on deep learning, which is applied to a roadside robot, and the system includes:

[0036] An acquisition module is used to obtain roadside environment information and dynamic image sequences of vehicles;

[0037] A feature recognition module, used to recognize static features and dynamic features of each target vehicle in the dynamic picture sequence based on the trained multi-target tracking model;

[0038] A screen image module is used to match images corresponding to each target state from the dynamic image sequence according to the static features and dynamic features of each target vehicle, so as to obtain a target image set of each target vehicle in each target state;

[0039] A frame recognition module, for recognizing the dynamic data of the target vehicle bounding box using the trained convolutional neural network model according to the target image set, and determining the target license plate bounding box based on the dynamic data of the target vehicle bounding box;

[0040] The generation module is used to perform optical character recognition on the target license plate boundary frame using an OCR recognition model to obtain the license plate information of the target vehicle, and generate roadside management information of the target vehicle in combination with the dynamic data of the target vehicle boundary frame and the roadside environment information.

[0041] In a third aspect, the present application provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to implement the roadside robot data processing method based on deep learning as described in the first aspect.

[0042] In a fourth aspect, the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed by a processor, the roadside robot data processing method based on deep learning described in the first aspect is implemented.

[0043] The roadside robot data processing method and system based on deep learning provided in this application have at least the following beneficial effects:

[0044] The technical solution provided by this application can comprehensively collect roadside environmental information and vehicle dynamic picture sequences at the information acquisition level, providing a rich data basis for subsequent analysis. The multi-target tracking model accurately identifies the static and dynamic characteristics of the vehicle, so that the roadside robot has a clear understanding of the vehicle. In the data processing link, the target picture set is obtained based on the feature matching picture, which provides strong support for vehicle status analysis. The convolutional neural network accurately identifies the dynamic data of the vehicle bounding box and determines the license plate bounding box, laying a solid foundation for license plate recognition. The OCR recognition model efficiently obtains license plate information, combines other information to generate roadside management information, and realizes all-round management of vehicles. It can adapt to complex roadside parking environments, effectively improve the efficiency of roadside traffic management, and effectively improve the data processing efficiency of road test robots. It can be effectively applied to complex roadside parking environments in cities at different sections and different time periods. By integrating data from various complex scenes during the training process, the model has good generalization capabilities and can cope with special traffic needs in different sections such as commercial areas, residential areas, and around schools, as well as roadside traffic conditions at different times such as daytime, night, peak hours, and low traffic hours. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the specific implementation methods or the prior art descriptions. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings without creative work. It should be noted that the drawings described below are schematic and should not be understood as any limitation to the present application. In the drawings:

[0046] Figure 1 A schematic diagram of a roadside robot data processing method based on deep learning in one embodiment of the present application is shown;

[0047] Figure 2 A schematic diagram of a roadside environment in one embodiment of the present application is shown;

[0048] Figure 3 A schematic diagram of a convolutional neural network model architecture in one embodiment of the present application is shown;

[0049] Figure 4 A schematic diagram of a roadside robot data processing system based on deep learning in one embodiment of the present application is shown;

[0050] Figure 5 A schematic diagram of a computer device in one embodiment of the present application is shown. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0052] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the system or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0053] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, it can also be the internal connection of two components, it can be a wireless connection, or it can be a wired connection. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0054] In addition, the technical features involved in the different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0055] Although the processes described below include multiple operations that appear in a specific order, it should be clearly understood that these processes may also include more or fewer operations, and these operations may be performed sequentially or in parallel.

[0056] Example 1

[0057] The roadside robot data processing method based on deep learning provided in this application can be applied to smart transportation application scenarios. In this scenario, the roadside robot performs efficient data processing and recognition of the entry, parking, and departure of road test vehicles based on the neural network model of deep learning, which is effectively applicable to complex road test parking environments in cities at different road sections and different time periods.

[0058] See also Figure 1, a roadside robot data processing method based on deep learning provided in one embodiment of the present application, applied to a roadside robot, may include the following multiple steps.

[0059] S101, obtaining roadside environment information and a dynamic picture sequence of a vehicle;

[0060] S102, identifying static features and dynamic features of each target vehicle in the dynamic picture sequence based on the trained multi-target tracking model;

[0061] S103, matching pictures corresponding to each target state from the dynamic picture sequence according to the static features and dynamic features of each target vehicle, to obtain a target picture set of each target vehicle in each target state;

[0062] S104, according to the target image set, using the trained convolutional neural network model to identify dynamic data of the target vehicle bounding box, and determining the target license plate bounding box based on the dynamic data of the target vehicle bounding box;

[0063] S105 , performing optical character recognition on the target license plate boundary frame using an OCR recognition model to obtain the license plate information of the target vehicle, and generating roadside management information of the target vehicle by combining the dynamic data of the target vehicle boundary frame and the roadside environment information.

[0064] In this embodiment, the roadside environment refers to Figure 2 As shown. By acquiring roadside environmental information and dynamic image sequences of vehicles, roadside robots can perceive surrounding traffic conditions in all directions. Specifically, roadside robots can be equipped with a variety of sensors to ensure the richness and accuracy of data, thus providing a basis for subsequent analysis and decision-making. For example, high-definition cameras, millimeter-wave radars, lidars and other sensors work together to obtain multi-dimensional information such as the vehicle's location, speed, and appearance, so that the roadside robot has a more comprehensive understanding of the current road test scenario.

[0065] Based on the trained multi-target tracking model, the static and dynamic features of each target vehicle in the dynamic image sequence can be accurately identified. This allows the roadside robot to distinguish different vehicles at the same time and conduct detailed analysis of the behavior of each vehicle. Whether it is the static features such as the color, brand, type, shape, or the dynamic features such as the position, speed, and direction of the vehicle, they can all be accurately identified, providing a key basis for subsequent vehicle matching and management.

[0066] According to the characteristics of the target vehicle, the pictures of the corresponding target state are matched from the dynamic picture sequence to obtain the target picture set of each target vehicle in different target states. This targeted data matching method can quickly and accurately filter out the picture information related to the specific target state, improving the efficiency and accuracy of data processing. For example, when judging whether the vehicle is in different states such as entering, parking or exiting, the corresponding pictures can be quickly found for analysis, which helps to grasp the behavior changes of the vehicle in a timely manner. When the vehicle enters, the front of the corresponding vehicle or the parking space direction is matched to the accurate picture of the high-definition camera state of the roadside robot according to the position and direction of the vehicle, so as to facilitate the subsequent effective identification of the license plate information of the vehicle. Reducing the data processing of invalid pictures improves the efficiency and accuracy of data processing.

[0067] The trained convolutional neural network model is used to identify the dynamic data of the target vehicle bounding box, and further determine the target license plate bounding box. The powerful feature extraction capability of the convolutional neural network makes the recognition of the vehicle bounding box and the license plate bounding box more accurate, and can adapt to the complex urban roadside parking environment. Whether under different lighting conditions or in the case of partial occlusion between vehicles, the position of the vehicle and the license plate can be accurately located, providing reliable protection for subsequent license plate recognition and management. It should be noted that, for example, when obtaining the target picture set in step S103, according to the static and dynamic features of each target vehicle, and in combination with the relationship between different vehicles, it is possible to avoid using the picture in the case of partial occlusion between vehicles as the target picture, that is, it is impossible to avoid partial occlusion between vehicles, and the corresponding auxiliary image can be collected later to identify the license plate information of the vehicle.

[0068] The OCR recognition model is used to perform optical character recognition on the target license plate bounding box, which can accurately obtain the license plate information of the target vehicle. Through training on a large number of license plate images, the OCR recognition model has a high recognition accuracy and robustness. Even if the license plate is stained, worn or partially blocked, it can recognize the license plate number as accurately as possible. It provides core support for the precise management of vehicles.

[0069] The roadside management information of the target vehicle is generated by combining the dynamic data of the target vehicle boundary box and the roadside environment information, realizing comprehensive management of the vehicle. The dynamic data of the target vehicle boundary box corresponds to the basic dynamic process of the vehicle from entering to leaving. The generated roadside management information covers all aspects of the vehicle's entry, parking, and departure, including important information such as the vehicle's driving trajectory, parking time, and whether it is illegal, which effectively improves the data processing efficiency of the road test robot.

[0070] In this embodiment, at the information acquisition level, roadside environmental information and vehicle dynamic picture sequences can be fully collected to provide a rich data basis for subsequent analysis. The multi-target tracking model accurately identifies the static and dynamic characteristics of the vehicle, so that the roadside robot has a clear understanding of the vehicle. In the data processing link, based on the feature matching picture, a target picture set is obtained to provide strong support for vehicle status analysis. The convolutional neural network accurately identifies the dynamic data of the vehicle bounding box and determines the license plate bounding box, laying a solid foundation for license plate recognition. The OCR recognition model efficiently obtains license plate information, combines other information to generate roadside management information, and realizes all-round management of vehicles. It can adapt to complex roadside parking environments, effectively improve the efficiency of roadside traffic management, and effectively improve the data processing efficiency of road test robots.

[0071] The method provided in this application can be effectively applied to complex roadside parking environments in cities at different road sections and different time periods. By incorporating data from various complex scenarios into the training process, the model has good generalization capabilities and can cope with special traffic needs in different road sections such as commercial areas, residential areas, and around schools, as well as traffic conditions at different times such as daytime, nighttime, peak hours, and low-traffic hours. This enables roadside robots to work stably and accurately in various practical scenarios, improving the overall reliability and practicality of the intelligent transportation system.

[0072] At the same time, through efficient data processing and identification of vehicles, the method provided by this application can realize intelligent management of roadside parking. For example, timely detection of illegal parking behaviors, optimization of parking space resource allocation, and improvement of vehicle traffic efficiency. This will help alleviate urban traffic congestion, improve the efficiency and level of urban traffic management, provide residents with a more convenient and orderly travel environment, and promote the development of smart transportation and the construction of smart cities.

[0073] Preferably, the roadside robot data processing method based on deep learning further includes:

[0074] When the target license plate bounding box cannot be determined from the dynamic data of the target vehicle bounding box, based on the current motion state of the robot, according to the dynamic data of the target vehicle bounding box, and in combination with the roadside environment information, the robot motion path and obstacle avoidance data for reaching the target vehicle are generated;

[0075] When the robot moves to the target vehicle based on the motion path and obstacle avoidance data, an auxiliary image of the target vehicle is obtained; wherein the target vehicle boundary box corresponding to the auxiliary image includes a license plate boundary box.

[0076] In this embodiment, when the target license plate bounding box cannot be determined from the dynamic data of the target vehicle bounding box, the robot motion path and obstacle avoidance data to reach the target vehicle can be generated based on the current motion state of the robot, combined with the dynamic data of the target vehicle bounding box and the roadside environment information. This decision-making mechanism reflects the flexibility and intelligence of the method of this application, and can provide a reasonable action plan for the roadside robot in the face of complex and changing actual situations to ensure the smooth progress of the task.

[0077] By comprehensively considering multiple factors to generate a motion path, the roadside robot can reach the target vehicle in the best or better way. During the movement, the robot can not only quickly approach the target, but also make real-time adjustments based on roadside environmental information (such as the location of obstacles, road layout, etc.) to avoid potential obstacles, thereby improving the efficiency and safety of movement. This efficient path planning capability helps to reduce the robot's action time and improve overall work efficiency, thereby better meeting real-time requirements.

[0078] When the robot moves to the target vehicle based on the motion path and obstacle avoidance data, the auxiliary image of the target vehicle is obtained, and the target vehicle boundary box corresponding to the auxiliary image contains the license plate boundary box. This design effectively solves the problem that the license plate boundary box cannot be determined in the initial data, that is, there is a partial occlusion between the above-mentioned vehicles, or the roadside robot is too far away from the incoming vehicle to take a clear picture, or other situations where the license plate information cannot be effectively identified. By actively acquiring auxiliary images, it is ensured that key data containing license plate information can be obtained, which provides the necessary conditions for subsequent license plate recognition and vehicle management. The process of acquiring auxiliary images further improves the data collection system of the roadside robot. In the complex urban roadside parking environment, due to various factors (such as vehicle occlusion, poor angle, etc.), the initially collected data may not meet the requirements of license plate recognition. The acquisition of auxiliary images through the autonomous movement of the robot can supplement the missing key information, ensure the integrity and accuracy of the data, and thus improve the reliability and stability of the entire data processing process.

[0079] In this embodiment, in order to solve the problem of poor real-time optimization capability of traditional path planning and obstacle avoidance methods, a path planning and obstacle avoidance method based on deep reinforcement learning can be adopted. Specifically, a deep Q network (DQN) algorithm is adopted, the state information (position, speed, direction, obstacle information, etc.) of the roadside robot is used as input, and the robot's action (forward, backward, left turn, right turn, stop, etc.) is used as output, and the optimal action strategy is learned through interaction with the environment.

[0080] The core of the DQN algorithm is to use a deep neural network to approximate the Q function, that is, Q(s,a)≈Q(s,a;θ), where θ represents the parameters of the neural network. The goal of the DQN algorithm is to learn the optimal Q function so that the agent can choose the optimal action in any state.

[0081] The update formula of the DQN algorithm is as follows:

[0082]

[0083] Where: θ i represents the neural network parameters of the i-th iteration; α represents the learning rate; R represents the reward; γ represents the discount factor; s represents the current state; a represents the action taken in the current state; s′ represents the next state; a′ represents the possible action taken in the next state; Represents the parameters of the target network, which are used to calculate the target Q value. The parameters of the target network are regularly copied from the current network to improve the stability of training.

[0084] The DQN algorithm uses the experience replay mechanism to store and sample experience data, breaking the correlation between data and improving training efficiency.

[0085] The method provided in this embodiment solves the challenge of being unable to determine the target license plate boundary box through the robot's autonomous action, thereby enhancing the reliability of the entire method. Even in complex environmental conditions, it can ensure that the required information can be obtained through self-adjustment and adaptation, reducing management errors caused by missing or inaccurate data and improving the efficiency, stability and reliability of roadside robot data processing.

[0086] Giving roadside robots the ability to make autonomous decisions and actions based on actual conditions greatly enhances the practical application value of roadside robots in smart traffic scenarios. It can not only be used for routine vehicle management tasks, but also play a role in special situations, providing a more comprehensive and efficient solution for traffic management. This will help promote the further popularization and development of smart traffic technology in practical applications, and bring greater convenience and benefits to urban traffic management.

[0087] Preferably, the trained multi-target tracking model identifies the static features and dynamic features of each target vehicle in the dynamic picture sequence, including:

[0088] Based on the trained multi-target tracking model, the data association algorithm is used to match and track the target vehicles in different frame images, and the static and dynamic features of each target vehicle are identified;

[0089] The static features include appearance color, category, and vehicle shape;

[0090] The dynamic features include the target vehicle's driving position coordinates, driving speed, and driving direction.

[0091] In this embodiment, a data association algorithm is used to match and track target vehicles in different frame images, which can ensure that the status of the same target vehicle at different times can be accurately associated in complex traffic scenes. This stable data association capability enables the roadside robot to continuously and accurately pay attention to the state changes of each target vehicle, avoiding the problem of target loss caused by vehicle movement and image frame switching, and providing a reliable basis for subsequent feature recognition and analysis. The multi-target tracking model combined with the data association algorithm can quickly and accurately track each target vehicle in a large number of vehicles and complex dynamic scenes. Even in the case of heavy traffic and frequent occlusion and intersection between vehicles, it can still maintain high tracking accuracy and efficiency, meeting the strict requirements of the intelligent transportation system for real-time and accuracy.

[0092] Clearly identifying the exterior color, category, vehicle shape, etc. as static features enables the roadside robot to distinguish and describe the target vehicle in more detail. It helps to quickly locate and identify specific vehicles, improves the recognition of vehicles, and provides more dimensional information for accurate vehicle management. The recognition of dynamic features such as the target vehicle's driving position coordinates, driving speed, and driving direction is used to match the corresponding vehicle status based on the dynamic feature information, thereby determining the corresponding key frame image from the dynamic image sequence, which is used for subsequent vehicle dynamic data recognition and license plate information recognition. It is the basis of the target image collection and further improves the data processing efficiency and applicability of the roadside robot.

[0093] Preferably, the matching of pictures corresponding to each target state from the dynamic picture sequence according to the static features and dynamic features of each target vehicle to obtain a target picture set of each target vehicle in each target state includes:

[0094] A feature matching algorithm is used to process each frame of the dynamic picture sequence. Based on the static feature matching of the target vehicle in the image, the dynamic features of the target vehicle are compared with the dynamic features of the corresponding target state to calculate the matching degree.

[0095] When the matching degree reaches a preset threshold, the frame image is added as a target image to the target image set corresponding to the target vehicle.

[0096] In this embodiment, a feature matching algorithm is used to process each frame of the dynamic picture sequence. On the basis of static feature matching, the dynamic features of the target vehicle are further compared with the dynamic features of the corresponding target state to calculate the matching degree. This matching method that comprehensively considers static and dynamic features greatly improves the accuracy of image matching. Static features such as appearance color, category, and vehicle shape provide basic identification information of the vehicle, while dynamic features such as driving position coordinates, driving speed, and driving direction reflect the real-time status of the vehicle. The combination of the two can more accurately determine whether a frame of image corresponds to a specific target state, avoiding the misjudgment that may occur in single feature matching.

[0097] By calculating the matching degree and comparing it with the preset threshold, it is decided whether to add the image to the target image set, which provides a quantitative standard for image matching. This makes the matching process more scientific and objective, and can flexibly adjust the strictness of the matching according to actual needs. For example, in scenarios with high matching accuracy requirements, the threshold can be appropriately increased; and in situations where more relevant images need to be obtained for comprehensive analysis, the threshold can be lowered. This adjustability enhances the adaptability and flexibility of the method and can meet the needs of different application scenarios.

[0098] The method provided in this embodiment can accurately select pictures corresponding to each target state from a large number of dynamic picture sequences, and form a target picture set of each target vehicle in different target states. This process realizes efficient data sorting and classification, making subsequent data processing and analysis more targeted, saving data retrieval and processing time, improving data processing efficiency, and improving the working efficiency of roadside robots.

[0099] By continuously matching and screening each frame of the image, it is possible to ensure that the target image collection covers the key information of the target vehicle in each target state, ensuring the integrity and consistency of the data. This is of great significance for in-depth research on vehicle behavior patterns, analysis of the changing laws of roadside vehicle volume, and tracing parking events. Complete and coherent data can provide a more comprehensive and accurate basis for roadside vehicle management and decision-making, and help to formulate more scientific and reasonable plans.

[0100] Preferably, the architecture of the trained convolutional neural network model includes multiple convolutional layers, pooling layers and fully connected layers;

[0101] Among them, the convolution layer uses convolution kernels of different sizes to extract the features of different states of the target vehicle;

[0102] The pooling layer uses a global average pooling method to reduce the dimension of the feature map output by the convolution layer and retain important features;

[0103] The fully connected layer uses a feature fusion network to fuse the features output by the pooling layer, and through the multi-target recognition layer, performs bounding box prediction and recognition on multiple target vehicles while predicting and recognizing their state categories;

[0104] The data set used to train the convolutional neural network model contains a large number of target vehicle images in different scenes, different angles, and different lighting conditions, and is annotated with precise vehicle bounding box position information;

[0105] During the training process, stochastic gradient descent and its variants are used, and the cross entropy loss function is combined with the regression loss function to optimize the model parameters.

[0106] In this embodiment, a convolutional neural network model architecture is as follows: Figure 3 As shown in the figure, the architecture of the convolutional neural network model includes multiple convolutional layers, and the convolutional layers use convolution kernels of different sizes to extract the features of different states of the target vehicle. Figure 3 The middle convolution layer includes the input layer, the first convolution layer, and the second convolution layer. This design enables the model to extract features of the target vehicle from multiple scales. Small-sized convolution kernels can capture detailed features of the vehicle, such as the texture of the license plate and the vehicle logo, while large-sized convolution kernels can obtain the overall features of the vehicle, such as the outline and general shape of the vehicle. By integrating features of different scales, the model can describe the target vehicle more comprehensively and accurately, improve the ability to recognize vehicles in different states, and effectively distinguish different vehicles and their states even in complex traffic scenes. The model is trained using a dataset containing a large number of target vehicle images in different scenes, different angles, and different lighting conditions, which makes the model highly adaptable. The diversity of different scenes (such as city streets, highways, parking lots, etc.), angles (front, side, back, etc.) and lighting conditions (strong light, weak light, backlight, etc.) allows the model to learn rich feature patterns. This means that in practical applications, the model can better cope with various complex real-world situations. Whether in different geographical locations, different times, or in various weather and lighting conditions, it can stably extract the features of the target vehicle, ensuring the generalization and robustness of the model.

[0107] Specifically, the convolution layer can be a depthwise separable convolution: decomposing the standard convolution into two steps: depthwise convolution and pointwise convolution. Depthwise convolution: convolution is performed on each input channel using a convolution kernel. Assuming the number of input channels is C in , the number of output channels is C out , the convolution kernel size is K×K, then the computational complexity of the depth convolution is:

[0108] Cost dw =K×K×C in ×H×W

[0109] Where H and W are the height and width of the feature map respectively.

[0110] Point-by-point convolution: Use a 1×1 convolution kernel to convolve the output of the depth convolution to fuse channel information. Assume that the number of output channels is C out , then the computational cost of point-by-point convolution is:

[0111] Cost pw =1×1×C in ×C out ×H×W

[0112] The total amount of calculation is:

[0113] Cost ds =Cost dw +Cost pw =K×K×C in ×H×W+C in ×C out ×H×W

[0114] Inverted Residual Block: First use 1×1 convolution to increase the dimension, then use deep convolution to extract features, and finally use 1×1 convolution to reduce the dimension, and add residual connections between the input and output.

[0115] In this embodiment, the pooling layer uses a global average pooling method to reduce the dimension of the feature map output by the convolution layer, which retains important features while reducing the amount of data. This dimensionality reduction operation not only reduces the computational complexity of the model, speeds up the running speed of the model, but also avoids the occurrence of overfitting. Through global average pooling, the model can extract representative features from the feature map, remove redundant information, make subsequent processing more efficient, and also ensure that the model accurately grasps the characteristics of the target vehicle. The fully connected layer uses a feature fusion network to fuse the features output by the pooling layer. In this way, the model can organically combine features of different levels and types, and further explore the intrinsic connection between features. This deep feature fusion helps to improve the expressive power of the model, so that the model can better understand the complex features of the target vehicle, thereby achieving better results in the multi-target recognition task. At the same time, through the multi-target recognition layer, the model can predict and recognize the state categories of multiple target vehicles while predicting and recognizing the bounding box, realizing comprehensive and accurate recognition of the target vehicle, and improving the practicality and efficiency of the model.

[0116] In this method, learnable weights are introduced to fuse features of different scales. For each feature fusion node, the calculation formula is as follows:

[0117]

[0118] in represents the i-th output feature, represents the jth input feature, w j represents the learnable weight of the j-th input feature.

[0119] In order to ensure that the weight is between 0 and 1 and the sum is 1, the softmax function is usually used to normalize the weight. The formula is

[0120] In this embodiment, the limitations of the intersection-over-union loss function (IoU Loss) are solved by the complete intersection-over-union loss function (CIoU Loss), which takes into account the center point distance and aspect ratio of the bounding box. The calculation formula of CIoU Loss is as follows:

[0121]

[0122] Where: "IoU" represents the intersection over union ratio between the predicted box and the true box; ρ(b,b gt ) represents the Euclidean distance between the center points of the predicted box and the real box; c represents the diagonal length of the minimum circumscribed rectangle of the predicted box and the real box; α is the weight coefficient; v is used to measure the consistency of the aspect ratio of the predicted box and the real box, and the calculation formula is:

[0123]

[0124] where w gt ,h gt Respectively represent the width and height of the real box, w and h represent the width and height of the predicted box respectively; the calculation formula of α is:

[0125]

[0126] In this embodiment, the fully connected layer uses a feature fusion network to fuse the features output by the pooling layer. In this way, the model can organically combine features of different levels and types, and further explore the intrinsic connection between features. This deep feature fusion helps to improve the expressiveness of the model, enabling the model to better understand the complex features of the target vehicle, thereby achieving better results in multi-target recognition tasks. At the same time, through the multi-target recognition layer, the model can predict and recognize the state categories of multiple target vehicles while predicting and recognizing the bounding box, achieving comprehensive and accurate recognition of the target vehicle, and improving the practicality and efficiency of the model.

[0127] During the training process, the stochastic gradient descent algorithm and its variants are used in combination with the cross entropy loss function and the regression loss function to optimize the model parameters. The stochastic gradient descent algorithm and its variants can converge quickly during the training process, find better model parameters, and improve training efficiency. The cross entropy loss function has a good optimization effect on classification tasks, and can effectively measure the difference between the category predicted by the model and the true category, thereby guiding the model to learn in the correct classification direction; the regression loss function is aimed at the bounding box prediction task, and can accurately adjust the model's prediction of the target vehicle's bounding box position, making the prediction result more accurate. This combined loss function design can simultaneously optimize the model's classification and positioning capabilities, and improve the overall performance of the model.

[0128] The training data set is annotated with precise vehicle bounding box position information, which provides accurate supervision signals for model training. During the training process, the model can continuously adjust its parameters based on this annotation information, making the predicted bounding box position closer and closer to the true value. Accurately annotated data is the key to the model's ability to accurately predict the target vehicle's bounding box, ensuring that the model can accurately locate the target vehicle in practical applications, providing a reliable foundation for subsequent license plate recognition and vehicle management.

[0129] Preferably, each frame of the target image set is processed independently, and the position coordinates, width information, and height information of the target vehicle boundary box in the frame of the image are output to form a dynamic data sequence of the target vehicle boundary box;

[0130] Based on the width information in the dynamic data of the target vehicle bounding box, a target frame image corresponding to the target width information is determined, and based on the trained convolutional neural network model, a target license plate bounding box in the target frame image is identified.

[0131] In this embodiment, each frame image in the target image set is processed independently, and the position coordinates, width information, and height information of the target vehicle boundary box are output to form a dynamic data sequence. The state changes of the target vehicle at different times are accurately recorded, providing rich and continuous data support for subsequent analysis. Through these dynamic data, the driving trajectory, stop position, and change information of the target vehicle can be clearly understood, which is helpful for in-depth analysis of the vehicle's behavior patterns and laws. Independently processing each frame image gives the system higher flexibility in data processing. When facing complex and changeable traffic scenes, targeted processing can be performed according to the specific circumstances of each frame image, avoiding errors and omissions that may occur due to the overall processing method. At the same time, this method is also convenient for separate analysis and processing of specific frame images, meeting the diverse needs for data in different application scenarios.

[0132] Based on the width information in the dynamic data of the target vehicle's bounding box, the target frame image corresponding to the target width information is determined, which can accurately screen out the frame images that are critical to license plate recognition from a large amount of image data. Different vehicles may have different bounding box widths in different states. In this way, the image that is most conducive to license plate recognition can be selected more specifically, improving the success rate and accuracy of license plate recognition.

[0133] After determining the target frame image, the trained convolutional neural network model is used to identify the target license plate bounding box. Since the convolutional neural network has a strong ability in image feature extraction, it can accurately identify the target license plate bounding box after a large amount of data training. Combined with the accurately screened target frame image, the efficiency and accuracy of license plate recognition are further improved, the recognition errors caused by image interference or unclear license plate features are reduced, and the data processing efficiency and applicability of the roadside robot are improved.

[0134] Preferably, the image within the license plate boundary frame is subjected to size normalization processing, and the size normalized image is subjected to grayscale conversion, noise reduction, binarization, and tilt correction preprocessing in sequence;

[0135] Extract local features of characters in preprocessed images, capture contextual relationships between characters, and obtain global feature information through self-attention mechanism;

[0136] When there are multiple candidate license plate information in the recognition result, they are screened according to the preset confidence threshold. When the confidence is higher than the threshold, the result with the highest confidence is selected as the final license plate information;

[0137] The generated roadside management information of the target vehicle includes: license plate information of the target vehicle, entry information, parking coordinate information, and exit information of the target vehicle.

[0138] In this embodiment, the image within the license plate boundary frame is normalized so that the license plate images of different sizes have a uniform size specification, which is convenient for subsequent feature extraction and model processing. After that, preprocessing operations such as grayscale, noise reduction, binarization and tilt correction are performed in sequence, which can significantly improve the quality of the image. Grayscale converts the color image into a grayscale image, simplifies the color information of the image, and reduces the amount of calculation; the noise reduction operation can remove the noise interference in the image and make the characters clearer; the binarization process converts the image into a black and white binary image to highlight the outline of the characters; the tilt correction corrects the tilt problem that may exist in the license plate image to ensure the integrity and accuracy of the characters. The comprehensive application of these preprocessing operations provides a high-quality image foundation for subsequent character feature extraction and recognition. After a series of preprocessing, the image has more prominent and stable features, which helps to improve the adaptability and accuracy of the subsequent recognition model. Regardless of the environmental factors that affect the license plate image during shooting (such as uneven lighting, angle deviation, etc.), the preprocessed image can restore the true features of the license plate characters as much as possible, so that the recognition model can better extract and analyze these features, thereby improving the reliability and stability of recognition.

[0139] Extracting local features of characters in the preprocessed image, capturing the contextual relationship between characters, and obtaining global feature information through the self-attention mechanism, this multi-dimensional feature extraction method can comprehensively and deeply mine the effective information in the license plate image. Local feature extraction can capture the detailed features of characters, such as the shape and length of strokes; capturing contextual relationships helps to understand the relationship between characters and further improve the accuracy of recognition; the self-attention mechanism can obtain global feature information, enabling the model to grasp the characteristics of the image as a whole and better handle complex license plate image situations. By combining these methods, the recognition ability of license plate characters can be improved, especially when facing license plate images with blurred, incomplete or complex backgrounds, key features can still be accurately extracted. By extracting and analyzing features at different levels and angles, the model can more accurately identify license plate characters and reduce misidentification. This comprehensive feature extraction method enables the model to make full use of various information in the license plate image, improves the recognition adaptability of different types of license plates, and thus can more accurately identify various license plate information in practical applications.

[0140] When there are multiple candidate license plate information in the recognition result, the preset confidence threshold is used for screening, and the result with the highest confidence higher than the threshold is selected as the final license plate information. This screening mechanism can effectively exclude candidate results with low confidence, ensuring the reliability of the final license plate information. By setting a reasonable confidence threshold, a balance can be found between recognition accuracy and recall rate, which not only avoids information errors caused by misidentification, but also covers all possible license plate information as much as possible. The screening process makes the recognition results more stable and reliable, which helps to improve the stability and reliability of the roadside robot.

[0141] Example 2

[0142] This embodiment provides a roadside robot data processing system based on deep learning, which is described by applying the roadside robot data processing method based on deep learning provided in the above embodiment 1. Figure 4 As shown, a roadside robot data processing system based on deep learning provided by one embodiment of the present application is applied to a roadside robot and may include the following multiple modules.

[0143] An acquisition module is used to obtain roadside environment information and dynamic image sequences of vehicles;

[0144] A feature recognition module, used to recognize static features and dynamic features of each target vehicle in the dynamic picture sequence based on the trained multi-target tracking model;

[0145] A screen image module is used to match images corresponding to each target state from the dynamic image sequence according to the static features and dynamic features of each target vehicle, so as to obtain a target image set of each target vehicle in each target state;

[0146] A frame recognition module, for recognizing the dynamic data of the target vehicle bounding box using the trained convolutional neural network model according to the target image set, and determining the target license plate bounding box based on the dynamic data of the target vehicle bounding box;

[0147] The generation module is used to perform optical character recognition on the target license plate boundary frame using an OCR recognition model to obtain the license plate information of the target vehicle, and generate roadside management information of the target vehicle in combination with the dynamic data of the target vehicle boundary frame and the roadside environment information.

[0148] The deep learning-based roadside robot data processing system provided in the embodiment of the present application can be applied to the deep learning-based roadside robot data processing method provided in the above-mentioned embodiment 1. For relevant details, refer to the above-mentioned method embodiment. The implementation principles and technical effects are similar and will not be repeated here.

[0149] It should be noted that: the roadside robot data processing system based on deep learning provided in the embodiments of the present application only uses the division of the above-mentioned functional modules / functional units as an example when performing roadside robot data processing based on deep learning. In actual applications, the above-mentioned functions can be assigned to different functional modules / functional units as needed, that is, the internal structure of the roadside robot data processing system based on deep learning is divided into different functional modules / functional units to complete all or part of the functions described above. In addition, the implementation method of the roadside robot data processing method based on deep learning provided in the above method embodiment 1 and the implementation method of the roadside robot data processing system based on deep learning provided in this embodiment 2 belong to the same concept. The specific implementation process of the roadside robot data processing system based on deep learning provided in this embodiment 2 is detailed in the above method embodiment 1, and will not be repeated here.

[0150] Example 3

[0151] See also Figure 5 As shown, one embodiment of the present application also provides a computer device, which can be a computer device such as a desktop computer, a laptop computer, a handheld computer, and a cloud server. The computer device can include, but is not limited to, a processor and a memory. The processor and the memory are connected via a bus or other means.

[0152] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, graphics processing units (GPU), embedded neural network processors (NPU) or other dedicated deep learning coprocessors, discrete gate or transistor logic devices, discrete hardware components and other chips, or a combination of the above chips.

[0153] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the methods in the above-mentioned embodiments of the present application. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, implementing the methods in the above-mentioned method embodiments.

[0154] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0155] One embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program. When the computer program is executed by a processor, the method in the above method embodiment is implemented.

[0156] Those skilled in the art can understand that the implementation of all or part of the processes in the above-mentioned implementation methods of the present application can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the implementation methods of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0157] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0158] Although the embodiments of the present application are described in conjunction with the accompanying drawings, they cannot be understood as limiting the scope of the patent application. It should be pointed out that for ordinary technicians in this field, other different forms of changes or modifications can be made on the basis of the above description without departing from the concept of the present application. It is not necessary and impossible to list all the embodiments here. The obvious changes or modifications derived from this are still within the scope of protection created by the present application. Therefore, the scope of protection of the patent application shall be based on the attached claims.

Claims

1. A roadside robot data processing method based on deep learning, characterized in that: The method comprises: Obtain roadside environment information and dynamic image sequences of vehicles; Identify static features and dynamic features of each target vehicle in the dynamic picture sequence based on the trained multi-target tracking model; According to the static features and dynamic features of each target vehicle, images corresponding to each target state are matched from the dynamic image sequence to obtain a target image set of each target vehicle in each target state; According to the target image set, a trained convolutional neural network model is used to identify dynamic data of a target vehicle bounding box, and based on the dynamic data of the target vehicle bounding box, a target license plate bounding box is determined; The target license plate boundary frame is subjected to optical character recognition using an OCR recognition model to obtain the license plate information of the target vehicle, and the roadside management information of the target vehicle is generated by combining the dynamic data of the target vehicle boundary frame and the roadside environment information.

2. The roadside robot data processing method based on deep learning according to claim 1 is characterized in that: Also includes: When the target license plate bounding box cannot be determined from the dynamic data of the target vehicle bounding box, based on the current motion state of the robot, according to the dynamic data of the target vehicle bounding box, and in combination with the roadside environment information, the robot motion path and obstacle avoidance data for reaching the target vehicle are generated; When the robot moves to the target vehicle based on the motion path and obstacle avoidance data, an auxiliary image of the target vehicle is obtained; wherein the target vehicle boundary box corresponding to the auxiliary image includes a license plate boundary box.

3. The roadside robot data processing method based on deep learning according to claim 2 is characterized in that: The multi-target tracking model based on the training is used to identify the static features and dynamic features of each target vehicle in the dynamic picture sequence, including: Based on the trained multi-target tracking model, the data association algorithm is used to match and track the target vehicles in different frame images, and the static and dynamic features of each target vehicle are identified; The static features include appearance color, category, and vehicle shape; The dynamic features include the target vehicle's driving position coordinates, driving speed, and driving direction.

4. The roadside robot data processing method based on deep learning according to claim 3 is characterized in that: The step of matching pictures corresponding to each target state from the dynamic picture sequence according to the static features and dynamic features of each target vehicle to obtain a target picture set of each target vehicle in each target state includes: A feature matching algorithm is used to process each frame of the dynamic picture sequence. Based on the static feature matching of the target vehicle in the image, the dynamic features of the target vehicle are compared with the dynamic features of the corresponding target state to calculate the matching degree. When the matching degree reaches a preset threshold, the frame image is added as a target image to the target image set corresponding to the target vehicle.

5. The roadside robot data processing method based on deep learning according to claim 4 is characterized in that: The architecture of the trained convolutional neural network model includes multiple convolutional layers, pooling layers and fully connected layers; Among them, the convolution layer uses convolution kernels of different sizes to extract the features of different states of the target vehicle; The pooling layer uses a global average pooling method to reduce the dimension of the feature map output by the convolution layer and retain important features; The fully connected layer uses a feature fusion network to fuse the features output by the pooling layer, and through the multi-target recognition layer, performs bounding box prediction and recognition on multiple target vehicles while predicting and recognizing their state categories; The data set used to train the convolutional neural network model contains a large number of target vehicle images in different scenes, different angles, and different lighting conditions, and is annotated with precise vehicle bounding box position information; During the training process, stochastic gradient descent and its variants are used, and the cross entropy loss function is combined with the regression loss function to optimize the model parameters.

6. The roadside robot data processing method based on deep learning according to claim 5 is characterized in that: Each frame of the target image set is processed independently, and the position coordinates, width information, and height information of the target vehicle boundary box in the frame of the image are output to form a dynamic data sequence of the target vehicle boundary box; Based on the width information in the dynamic data of the target vehicle bounding box, a target frame image corresponding to the target width information is determined, and based on the trained convolutional neural network model, a target license plate bounding box in the target frame image is identified.

7. The roadside robot data processing method based on deep learning according to claim 6 is characterized in that: Performing size normalization processing on the image within the license plate boundary frame, and sequentially performing grayscale conversion, noise reduction, binarization, and tilt correction preprocessing on the size normalized image; Extract local features of characters in preprocessed images, capture contextual relationships between characters, and obtain global feature information through self-attention mechanism; When there are multiple candidate license plate information in the recognition result, they are screened according to the preset confidence threshold. When the confidence is higher than the threshold, the result with the highest confidence is selected as the final license plate information; The generated roadside management information of the target vehicle includes: license plate information of the target vehicle, entry information, parking coordinate information, and exit information of the target vehicle.

8. A roadside robot data processing system based on deep learning, characterized in that: The system comprises: An acquisition module is used to obtain roadside environment information and dynamic image sequences of vehicles; A feature recognition module, used to recognize static features and dynamic features of each target vehicle in the dynamic picture sequence based on the trained multi-target tracking model; A screen image module is used to match images corresponding to each target state from the dynamic image sequence according to the static features and dynamic features of each target vehicle, so as to obtain a target image set of each target vehicle in each target state; A frame recognition module, for recognizing the dynamic data of the target vehicle bounding box using the trained convolutional neural network model according to the target image set, and determining the target license plate bounding box based on the dynamic data of the target vehicle bounding box; The generation module is used to perform optical character recognition on the target license plate boundary frame using an OCR recognition model to obtain the license plate information of the target vehicle, and generate roadside management information of the target vehicle in combination with the dynamic data of the target vehicle boundary frame and the roadside environment information.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor implements the roadside robot data processing method based on deep learning as described in any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the roadside robot data processing method based on deep learning as described in any one of claims 1 to 7 is implemented.

Images