Method and device for detecting illegal behavior of electric bicycles

Abstract

The present application provides a method and device for detecting violations of electric bicycles, the method for detecting violations of electric bicycles comprising: obtaining a lane line image taken by a camera device on the electric bicycle; identifying the lane line in the lane line image to obtain first position information of the lane line on the lane line image; obtaining second position information of a preset reference point of the lane line image based on the lane line image; and determining the violation of the electric bicycle based on the first position information and the second position information. The present application uses a camera device disposed on the electric bicycle to capture the lane line image, identify the position of the lane line, and then determine the violation according to the position of the lane line and the preset reference point. The violation can be determined without identifying the position of the electric bicycle, thereby avoiding the influence of the electric bicycle position identification error on the detection result and improving the accuracy of the method for detecting violations of electric bicycles.

Description

Technical Field

[0001] The present application relates to the technical field of electric vehicle violation detection, and specifically to a method and device for detecting violation of an electric bicycle. Background Art

[0002] In recent years, electric bicycles have rapidly become popular in my country and have gradually become an important means of transportation for people to travel short and medium distances. At present, there are about 200 million electric bicycles in the society. As a result, some drivers have no awareness of safe driving, occupying motor vehicle lanes, running red lights, driving in the opposite direction, illegally carrying passengers, driving under the influence of alcohol, operating illegally, adding canopies, and passing through prohibited areas. As a result, the traffic accident rate of electric bicycles has been rising year by year. Due to the lack of corresponding traffic management rules and measures, electric bicycles have become one of the culprits of urban traffic chaos.

[0003] In the existing technology, the detection of whether an electric bicycle has violated the rules mainly relies on static cameras deployed on public roads, various parks, main roads of highways, and important entrances and exits in the city to collect images of violations and then determine the violations. This solution requires identifying the positions of the electric bicycle and the lane lines, and then judging whether the electric bicycle has violated the rules. However, there are errors in identifying the position of the electric bicycle by manual or machine, and the existing solutions have low accuracy.

[0004] That is, in the prior art, the detection method of electric bicycle violations has low accuracy. Summary of the invention

[0005] The present application provides a method and device for detecting violations of electric bicycles, aiming to solve the problem of low accuracy of the method for detecting violations of electric bicycles.

[0006] In one aspect, the present application provides a detection method, comprising:

[0007] Acquire a lane line image captured by a camera device on the electric bicycle;

[0008] Identify a lane line in the lane line image to obtain first position information of the lane line on the lane line image;

[0009] Acquire second position information of a preset reference point of the lane line image based on the lane line image;

[0010] A violation of the electric bicycle is determined based on the first location information and the second location information.

[0011] The identifying of the lane line in the lane line image to obtain first position information of the lane line on the lane line image includes:

[0012] Semantic segmentation is performed on the lane line image to obtain first position information of the lane line on the lane line image.

[0013] The performing semantic segmentation on the lane line image to obtain first position information of the lane line on the lane line image includes:

[0014] Performing image classification on the lane line image based on a classification neural network model with an attention mechanism to obtain at least two classification results of the lane line image and at least two classification confidences corresponding to the at least two classification results;

[0015] If all the classification confidences of the at least two classification confidences are less than a preset value, semantic segmentation is performed on the lane line image to obtain first position information of the lane line on the lane line image.

[0016] The performing image classification on the lane line image based on the classification neural network model with an attention mechanism to obtain at least two classification results of the lane line image and at least two classification confidences corresponding to the at least two classification results includes:

[0017] Extracting a first feature map of the lane line image;

[0018] Redistributing the weight of the first feature map based on the attention mechanism to obtain a redistributed second feature map;

[0019] The lane line image is classified based on the second feature map to obtain at least two classification results of the lane line image and at least two classification confidences corresponding to the at least two classification results.

[0020] Wherein, the detection method further comprises:

[0021] If there is a classification confidence that is not less than a preset value among the at least two classification confidences, the classification result with the largest classification confidence among the at least two classification results is determined as the target classification result;

[0022] The violation of the electric bicycle is determined based on the target classification result.

[0023] Wherein, the preset reference point is the image center point of the lane line image, the lane line includes a first lane line and a second lane line; the first position information includes third position information of the first lane line on the lane line image and fourth position information of the second lane line on the lane line image;

[0024] The determining the violation of the electric bicycle based on the first location information and the second location information includes:

[0025] Determine a first relative position relationship between the first lane line and the center point of the image based on the third position information and the second position information;

[0026] If the first relative position relationship satisfies a first preset condition, determining a second relative position relationship between the second lane line and the center point of the image based on the fourth position information and the second position information;

[0027] The violation of the electric bicycle is determined based on the second relative position relationship.

[0028] Wherein, the detection method further comprises:

[0029] If the first relative position relationship does not satisfy the first preset condition, it is determined that the illegal behavior of the electric bicycle is wrong-way driving.

[0030] On the one hand, the present application provides a detection device for electric bicycle violation behavior, the detection device comprising:

[0031] A first acquisition unit is used to acquire a lane line image taken by a camera device on the electric bicycle;

[0032] A lane line recognition unit, used to recognize a lane line in the lane line image and obtain first position information of the lane line on the lane line image;

[0033] A second acquisition unit, configured to acquire second position information of a preset reference point of the lane line image based on the lane line image;

[0034] A determination unit is used to determine the violation of the electric bicycle based on the first position information and the second position information.

[0035] Wherein, the lane line recognition unit is further used to perform semantic segmentation on the lane line image to obtain first position information of the lane line on the lane line image.

[0036] The lane line recognition unit is further used to perform image classification on the lane line image based on a classification neural network model with an attention mechanism to obtain at least two classification results of the lane line image and at least two classification confidences corresponding to the at least two classification results;

[0037] If all the classification confidences of the at least two classification confidences are less than a preset value, semantic segmentation is performed on the lane line image to obtain first position information of the lane line on the lane line image.

[0038] Wherein, the lane line recognition unit is further used to extract a first feature map of the lane line image;

[0039] Redistributing the weight of the first feature map based on the attention mechanism to obtain a redistributed second feature map;

[0040] The lane line image is classified based on the second feature map to obtain at least two classification results of the lane line image and at least two classification confidences corresponding to the at least two classification results.

[0041] The determining unit is further configured to determine the classification result with the largest classification confidence among the at least two classification results as the target classification result if there is a classification confidence that is not less than a preset value among the at least two classification confidences;

[0042] The violation of the electric bicycle is determined based on the target classification result.

[0043] Wherein, the preset reference point is the image center point of the lane line image, the lane line includes a first lane line and a second lane line; the first position information includes third position information of the first lane line on the lane line image and fourth position information of the second lane line on the lane line image;

[0044] The determining unit is further used to determine a first relative position relationship between the first lane line and the center point of the image based on the third position information and the second position information;

[0045] If the first relative position relationship satisfies a first preset condition, determining a second relative position relationship between the second lane line and the center point of the image based on the fourth position information and the second position information;

[0046] The violation of the electric bicycle is determined based on the second relative position relationship.

[0047] Wherein, the determination unit is further used to determine that the illegal behavior of the electric bicycle is wrong-way driving if the first relative position relationship does not meet a first preset condition.

[0048] In one aspect, the present application further provides an electronic device, the electronic device comprising:

[0049] one or more processors;

[0050] Memory; and

[0051] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the method for detecting electric bicycle violations described in any one of the first aspects.

[0052] On the one hand, the present application also provides a computer-readable storage medium on which a computer program is stored, and the computer program is loaded by a processor to execute the steps in the method for detecting violations of electric bicycles described in any one of the first aspects.

[0053] The present application provides a method for detecting violations of electric bicycles. A camera device installed on the electric bicycle captures an image of a lane line, identifies the position of the lane line, and then determines the violation based on the position of the lane line and a preset reference point. The violation can be determined without identifying the position of the electric bicycle, thereby avoiding the influence of the error in identifying the position of the electric bicycle on the detection result and improving the accuracy of the method for detecting violations of electric bicycles. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0055] Figure 1 A schematic diagram of a scenario of a detection system for electric bicycle violations provided in an embodiment of the present application;

[0056] Figure 2 It is a flow chart of an embodiment of a method for detecting illegal behavior of an electric bicycle provided in an embodiment of the present application;

[0057] Figure 3 It is a schematic diagram of the structure of an embodiment of a detection device for electric bicycle violation provided in an embodiment of the present application;

[0058] Figure 4 It is a schematic diagram of the structure of an embodiment of an electronic device provided in the embodiments of the present application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0060] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or relative position relationships based on the positions or relative position relationships shown in the drawings, which are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0061] In this application, the word "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any technician in the field to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0062] It should be noted that since the method of the embodiment of the present application is executed in an electronic device, the processing objects of each electronic device exist in the form of data or information. For example, time is actually time information. It can be understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data for processing by the electronic device. The details will not be elaborated here.

[0063] The embodiments of the present application provide a method and device for detecting violations of electric bicycles, which are described in detail below.

[0064] See also Figure 1 , Figure 1 A schematic diagram of a scenario of an electric bicycle violation detection system provided in an embodiment of the present application. The electric bicycle violation detection system may include an electronic device 100, in which an electric bicycle violation detection device is integrated, such as Figure 1 Electronic devices in.

[0065] In the embodiment of the present application, the electronic device 100 is mainly used to obtain a lane line image taken by a camera device on an electric bicycle; identify the lane lines in the lane line image to obtain first position information of the lane lines on the lane line image; obtain second position information of a preset reference point of the lane line image based on the lane line image; and determine the violation of the electric bicycle based on the first position information and the second position information.

[0066] In the embodiment of the present application, the electronic device 100 may be an independent server, or a server network or server cluster composed of servers. For example, the electronic device 100 described in the embodiment of the present application includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets or a cloud server composed of a plurality of servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0067] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1 More or fewer electronic devices as shown in, e.g. Figure 1 Only one electronic device is shown. It can be understood that the electric bicycle violation detection system can also include one or more other services, which are not limited here.

[0068] In addition, if Figure 1 As shown, the electric bicycle violation detection system may further include a memory 200 for storing data.

[0069] It should be noted that Figure 1 The scenario diagram of the electric bicycle violation detection system shown is merely an example. The electric bicycle violation detection system and scenario described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art can appreciate that with the evolution of the electric bicycle violation detection system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is equally applicable to similar technical problems.

[0070] First, an embodiment of the present application provides a method for detecting violations of electric bicycles, which includes: obtaining a lane line image taken by a camera device on the electric bicycle; identifying the lane lines in the lane line image to obtain first position information of the lane lines on the lane line image; obtaining second position information of a preset reference point of the lane line image based on the lane line image; and determining the violation of the electric bicycle based on the first position information and the second position information.

[0071] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of an embodiment of a method for detecting illegal behavior of an electric bicycle provided in an embodiment of the present application. Figure 2 As shown, the detection method includes:

[0072] S201. Acquire a lane line image captured by a camera device on the electric bicycle.

[0073] In the embodiment of the present application, the camera device can be a camera, a mobile phone, a driving recorder, etc. fixed on the electric bicycle, which can shoot the road conditions in front of the electric bicycle in real time to obtain a lane line video. Specifically, the detection device for electric bicycle violation behavior obtains the lane line video shot by the camera device on the electric bicycle, decodes the lane line video at a preset frequency, and extracts the lane line image. Among them, the preset frequency can be 1hz, 0.5hz, etc., which can be specifically set according to the computing power of the detection device.

[0074] S202: Identify the lane line in the lane line image to obtain first position information of the lane line on the lane line image.

[0075] In an embodiment of the present application, semantic segmentation is performed on the lane line image to obtain the first position information of the lane line on the lane line image. Semantic segmentation is a basic task in computer vision. In semantic segmentation, we need to divide the visual input into different semantically interpretable categories. The semantic interpretability, that is, the classification category, is meaningful in the real world. The process of semantic segmentation can be understood as performing classification on each pixel of the image, and labeling each pixel in the image with a category, such as dividing the image into labels such as people (red) and lane lines (green). Of course, instance segmentation can also be performed on the lane line to obtain the first position information of the lane line on the lane line image.

[0076] Specifically, the lane line image can be semantically segmented by a lightweight semantic segmentation network model such as DABNet, cgnet (Context Guided Network), bisenet (Bilatel Segmentation Network), etc., to obtain each pixel point identified as a lane line on the lane line image, thereby obtaining the first position information of the lane line on the lane line image. The first position information can be the coordinates of the first lane line. By establishing a coordinate system in the lane line image, the coordinates of each pixel point of the first lane line can be obtained.

[0077] Furthermore, the lightweight semantic segmentation network model is a trained lightweight semantic segmentation network model.

[0078] DABNet (Depth-wise Asymmetric Bottleneck DAB module) efficiently uses deep asymmetric convolution and dilated convolution to build a bottleneck structure. DABNet built with DAB module can construct sufficient receptive field and densely utilize context information, and can achieve high accuracy without any pre-training and post-processing. The network first uses three 3x3 convolutions to extract the initial features. The initial downsampling strategy is a concat of a 3x3 convolution with a stride of 2 and a 2x2 maximum pooling. The rest of the downsampling stages are all 3x3 convolutions with a stride of 2 to make the downsampling rate not particularly large. In the downsampling stage, the original image is concatenates to the downsampling module to promote feature reuse and compensate for information loss. In order to better strengthen spatial relationships and feature propagation, inter-block cascading is introduced to combine high-level features with low-level features, which means stacking the first DAB module and the last DAB module in each DAB block. For dilated convolutions, all DAB modules in DAB block 1 include depthwise asymmetric dilated convolutions with a dilation rate of 2, and the dilation rates in DAB block 2 are 4, 4, 8, 8, 16, 16 respectively.

[0079] In a specific embodiment, semantic segmentation is performed on the lane line image to obtain first position information of the lane line on the lane line image, and the following steps may also be included:

[0080] (1) Performing image classification on the lane line image based on a classification neural network model with an attention mechanism to obtain at least two classification results of the lane line image and at least two classification confidences corresponding to the at least two classification results.

[0081] Specifically, a first feature map of the lane line image is extracted; the weight of the first feature map is redistributed based on the attention mechanism to obtain a redistributed second feature map; the lane line image is classified based on the second feature map to obtain at least two classification results of the lane line image and at least two classification confidences corresponding to the at least two classification results.

[0082] For example, at least two classification results are wrong-way driving on a motor vehicle lane, occupying a motor vehicle lane, and wrong-way driving on a non-motor vehicle lane. After the lane line image is classified by the classification neural network model with an attention mechanism, the classification results are: occupying a motor vehicle lane, classification confidence 0.2; wrong-way driving on a motor vehicle lane, classification confidence 0.7; wrong-way driving on a non-motor vehicle lane, classification confidence 0.1.

[0083] The classification neural network model with attention mechanism can be an Efficientnet classification network model with attention mechanism. The basic network architecture of Efficientnet (Rethinking Model Scaling for Convolutional Neural Networks) is designed by using neural architecture search. A simple and efficient composite coefficient is used to weigh the network depth, width and input image resolution.

[0084] Specifically, the classification neural network model with an attention mechanism can be trained by using lane line images marked with violations to obtain a trained classification neural network model with an attention mechanism, and the trained classification neural network model with an attention mechanism can be used to perform image classification on lane line images.

[0085] In one embodiment of the present application, the Efficientnet classification network model with attention mechanism can be an Efficientnet-b0 structure, including multiple connected convolutional layers and fully connected layers. An attention mechanism module is introduced at the end of the convolutional layer to redistribute image features and automatically obtain the importance of each feature channel through learning. The module includes a global average pooling layer, a 1*1 convolutional layer, a batch normalization layer, and a Sigmoid. After the attention mechanism module, the fully connected layer is connected, and the classification confidence of the corresponding violation is obtained through softmax.

[0086] (2) If all classification confidences of at least two classification confidences are less than a preset value, semantic segmentation is performed on the lane line image to obtain first position information of the lane line on the lane line image.

[0087] The preset value may be 0.7, 0.8, etc. If all the classification confidences of at least two classification confidences are less than the preset value, it indicates that the classification neural network model with the attention mechanism cannot provide a prediction result with a high confidence level, and each prediction result is unreliable, so it is necessary to make a prediction again, and then the lane line image is semantically segmented to obtain the first position information of the lane line on the lane line image. That is, when the initial prediction result does not meet the requirements, the lane line image is predicted again through more accurate semantic segmentation, and the lane line image suspected of violation can be re-judged, thereby improving the accuracy of determining the violation of the electric bicycle.

[0088] Furthermore, if there is a classification confidence that is not less than a preset value among at least two classification confidences, the classification result with the largest classification confidence among the at least two classification results is determined as the target classification result; and the violation of the electric bicycle is determined based on the target classification result. For example, occupying the motor vehicle lane, the classification confidence is 0.05; driving against the flow on the motor vehicle lane, the classification confidence is 0.85; driving against the flow on the non-motor vehicle lane; the classification confidence is 0.1, and the preset value is 0.8, then the target classification result is driving against the flow on the motor vehicle lane, and driving against the flow on the motor vehicle lane is determined as a violation of the electric bicycle. That is, the violation of the electric bicycle is initially predicted by a classification neural network model with an attention mechanism, and when the result of the initial prediction meets the requirements, the violation is directly determined based on the predicted classification result, thereby improving the detection efficiency.

[0089] S203. Acquire second position information of a preset reference point of the lane line image based on the lane line image.

[0090] In the embodiment of the present application, the preset reference point can be a preset pixel point on the lane line image, and specifically can be the image center point of the lane line image. Of course, the preset reference point can also be adjusted according to the installation position of the camera device relative to the electric bicycle, and the present application does not limit this. Specifically, the second position information is the coordinates of the center point of the lane line image.

[0091] S204. Determine the violation of the electric bicycle based on the first location information and the second location information.

[0092] In the embodiment of the present application, the lane line includes a first lane line and a second lane line. For example, the first lane line is a solid yellow lane line and the second lane line is a solid white lane line. Of course, the first lane line can also be a dividing strip or guardrail in the middle of the road, and the second lane line can also be a virtual white lane line, which can be determined according to different road conditions. The first position information includes the third position information of the first lane line on the lane line image and the fourth position information of the second lane line on the lane line image. The third position information is the coordinate of the first lane line on the lane line image; the fourth position information is the coordinate of the second lane line on the lane line image.

[0093] In one specific embodiment, the violation of an electric bicycle may be: occupying a motor vehicle lane and driving against the flow. In another specific embodiment, the violation of an electric bicycle may be: occupying a motor vehicle lane, driving against the flow on a motor vehicle lane, and driving against the flow on a non-motor vehicle lane, that is, the violation of driving against the flow is further subdivided into driving against the flow on a motor vehicle lane and driving against the flow on a non-motor vehicle lane.

[0094] In a specific embodiment, determining the violation of the electric bicycle based on the first location information and the second location information includes:

[0095] (1) Determine a first relative position relationship between the first lane line and the center point of the image based on the third position information and the second position information.

[0096] For example, the first relative position relationship is that the center point of the image is located on the left side of the first lane line, or the center point of the image is located on the right side of the first lane line.

[0097] Specifically, a target pixel point in the first lane line with the same horizontal coordinate as the image center point is obtained from the first position information. If the image center point is located on the left side of the target pixel point, it is determined that the image center point is located on the left side of the first lane line; if the image center point is located on the right side of the target pixel point, it is determined that the image center point is located on the right side of the first lane line.

[0098] (2) If the first relative position relationship satisfies the first preset condition, determining a second relative position relationship between the second lane line and the center point of the image based on the fourth position information and the second position information.

[0099] The first preset condition may be a preset condition indicating a positional relationship between the first lane line and the center point of the image, and specifically, the center point of the image may be located on the left side of the first lane line. Specifically, the first preset condition may be determined according to local traffic regulations. For example, in China's traffic regulations, it is normal for a vehicle to drive on the right side of the road.

[0100] When the first relative position relationship meets the first preset condition, it indicates that the electric bicycle is not in a reverse state. At this time, it is necessary to further judge the violation of the electric bicycle based on the second lane line. Specifically, the second relative position relationship between the second lane line and the center point of the image is determined based on the fourth position information and the second position information. For example, the second relative position relationship may include: the center point of the image is located on the left side of the second lane line, and there is only one second lane line located on the left side of the center point of the image. The determination of the second relative position relationship can refer to the determination method of the first relative position relationship.

[0101] (3) Determine the violation of the electric bicycle based on the second relative position relationship.

[0102] For example, if the center point of the image is located on the left side of the second lane line, the corresponding electric bicycle's violation is occupying the motor vehicle lane; if there is only one second lane line located on the left side of the center point of the image, the corresponding electric bicycle is driving normally.

[0103] Furthermore, when the first relative position relationship does not meet the first preset condition, it indicates that the electric bicycle is in a reverse state, and the violation of the electric bicycle is determined to be reverse. Furthermore, when the violation of the electric bicycle is determined to be reverse, it is determined based on the second relative position relationship whether the violation of the electric bicycle is reverse in a non-motor vehicle lane or reverse in a motor vehicle lane. For example, when the violation of the electric bicycle is determined to be reverse, if there is only one second lane line located to the right of the center point of the image, then the violation of the electric bicycle is reverse in the non-motor vehicle lane; if the center point of the image is located to the right of the second lane line, then the violation of the electric bicycle is reverse in the motor vehicle lane.

[0104] The present application provides a method for detecting violations of electric bicycles. A camera device installed on the electric bicycle captures an image of a lane line, identifies the position of the lane line, and then determines the violation based on the position of the lane line and a preset reference point. The violation can be determined without identifying the position of the electric bicycle, thereby avoiding the influence of the error in identifying the position of the electric bicycle on the detection result and improving the accuracy of the method for detecting violations of electric bicycles.

[0105] In order to better implement the electric bicycle violation detection method in the embodiment of the present application, based on the electric bicycle violation detection method, the embodiment of the present application also provides an electric bicycle violation detection device, such as Figure 3 As shown, Figure 3 : is a schematic diagram of an embodiment of a detection device for electric bicycle violation provided in an embodiment of the present application, and the detection device for electric bicycle violation includes:

[0106] The first acquisition unit 301 is used to acquire a lane line image taken by a camera device on the electric bicycle;

[0107] A lane line recognition unit 302 is used to recognize a lane line in the lane line image and obtain first position information of the lane line on the lane line image;

[0108] A second acquisition unit 303, configured to acquire second position information of a preset reference point of the lane line image based on the lane line image;

[0109] The determination unit 304 is used to determine the violation of the electric bicycle based on the first position information and the second position information.

[0110] The lane line recognition unit 302 is further used to perform semantic segmentation on the lane line image to obtain first position information of the lane line on the lane line image.

[0111] The lane line recognition unit 302 is further used to perform image classification on the lane line image based on a classification neural network model with an attention mechanism to obtain at least two classification results of the lane line image and at least two classification confidences corresponding to the at least two classification results;

[0112] If all classification confidences of at least two classification confidences are less than a preset value, semantic segmentation is performed on the lane line image to obtain first position information of the lane line on the lane line image.

[0113] The lane line recognition unit 302 is also used to extract a first feature map of the lane line image;

[0114] Redistribute the weight of the first feature map based on the attention mechanism to obtain a redistributed second feature map;

[0115] The lane line image is classified based on the second feature map to obtain at least two classification results of the lane line image and at least two classification confidences corresponding to the at least two classification results.

[0116] The determining unit 304 is further configured to determine the classification result with the largest classification confidence among the at least two classification results as the target classification result if there is a classification confidence that is not less than a preset value among the at least two classification confidences;

[0117] Determine the violations of electric bicycles based on the target classification results.

[0118] The preset reference point is the image center point of the lane line image, and the lane line includes a first lane line and a second lane line; the first position information includes third position information of the first lane line on the lane line image and fourth position information of the second lane line on the lane line image;

[0119] The determination unit 304 is further configured to determine a first relative position relationship between the first lane line and the center point of the image based on the third position information and the second position information;

[0120] If the first relative position relationship satisfies the first preset condition, determining a second relative position relationship between the second lane line and the center point of the image based on the fourth position information and the second position information;

[0121] The violation of the electric bicycle is determined based on the second relative position relationship.

[0122] The determination unit 304 is further configured to determine that the violation of the electric bicycle is wrong-way driving if the first relative position relationship does not satisfy a first preset condition.

[0123] The present application also provides an electronic device that integrates any of the electric bicycle violation detection devices provided in the present application. Figure 4 As shown, it shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:

[0124] The electronic device may include components such as a processor 601 with one or more processing cores, a memory 602 with one or more computer-readable storage media, a power supply 603, and an input unit 604. Those skilled in the art will appreciate that the electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0125] The processor 601 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 602, and calling data stored in the memory 602, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 601 may include one or more processing cores; preferably, the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 601.

[0126] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.

[0127] The electronic device also includes a power supply 603 for supplying power to each component. Preferably, the power supply 603 can be logically connected to the processor 601 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 603 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0128] The electronic device may further include an input unit 604, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0129] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 601 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 602 according to the following instructions, and the processor 601 will run the application programs stored in the memory 602, thereby realizing various functions, as follows:

[0130] Acquire a lane line image taken by a camera device on an electric bicycle; identify the lane lines in the lane line image to obtain first position information of the lane lines on the lane line image; acquire second position information of a preset reference point of the lane line image based on the lane line image; and determine the violation of the electric bicycle based on the first position information and the second position information.

[0131] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0132] To this end, the embodiment of the present application provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any of the electric bicycle violation detection methods provided in the embodiment of the present application. For example, the computer program is loaded by a processor to execute the following steps:

[0133] Acquire a lane line image taken by a camera device on an electric bicycle; identify the lane lines in the lane line image to obtain first position information of the lane lines on the lane line image; acquire second position information of a preset reference point of the lane line image based on the lane line image; and determine the violation of the electric bicycle based on the first position information and the second position information.

[0134] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above, and will not be repeated here.

[0135] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments, which will not be repeated here.

[0136] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0137] The above is a detailed introduction to the method and device for detecting violations of electric bicycles provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for detecting illegal behavior of electric bicycles, characterized in that: The detection method comprises: Acquire a lane line image captured by a camera device on the electric bicycle; Identify a lane line in the lane line image to obtain first position information of the lane line on the lane line image; Acquire second position information of a preset reference point of the lane line image based on the lane line image; Determine a violation of the electric bicycle based on the first location information and the second location information; Wherein, the preset reference point is the image center point of the lane line image, the lane line includes a first lane line and a second lane line; the first position information includes third position information of the first lane line on the lane line image and fourth position information of the second lane line on the lane line image; The determining the violation of the electric bicycle based on the first location information and the second location information includes: Determine a first relative position relationship between the first lane line and the center point of the image based on the third position information and the second position information; If the first relative position relationship satisfies a first preset condition, determining a second relative position relationship between the second lane line and the center point of the image based on the fourth position information and the second position information; The violation of the electric bicycle is determined based on the second relative position relationship.

2. The detection method according to claim 1, characterized in that The identifying the lane line in the lane line image to obtain first position information of the lane line on the lane line image includes: Semantic segmentation is performed on the lane line image to obtain first position information of the lane line on the lane line image.

3. The detection method according to claim 2, characterized in that The performing semantic segmentation on the lane line image to obtain first position information of the lane line on the lane line image includes: Performing image classification on the lane line image based on a classification neural network model with an attention mechanism to obtain at least two classification results of the lane line image and at least two classification confidences corresponding to the at least two classification results; If all the classification confidences of the at least two classification confidences are less than a preset value, semantic segmentation is performed on the lane line image to obtain first position information of the lane line on the lane line image.

4. The detection method according to claim 3, characterized in that The lane line image is classified based on the classification neural network model with an attention mechanism to obtain at least two classification results of the lane line image and at least two classification confidences corresponding to the at least two classification results, including: Extracting a first feature map of the lane line image; Redistributing the weight of the first feature map based on the attention mechanism to obtain a redistributed second feature map; The lane line image is classified based on the second feature map to obtain at least two classification results of the lane line image and at least two classification confidences corresponding to the at least two classification results.

5. The detection method according to claim 3, characterized in that: The detection method further comprises: If there is a classification confidence that is not less than a preset value among the at least two classification confidences, the classification result with the largest classification confidence among the at least two classification results is determined as the target classification result; The violation of the electric bicycle is determined based on the target classification result.

6. The detection method according to claim 1, characterized in that The detection method further comprises: If the first relative position relationship does not satisfy the first preset condition, it is determined that the illegal behavior of the electric bicycle is wrong-way driving.

7. A detection device for electric bicycle violations, characterized in that: The detection device comprises: A first acquisition unit is used to acquire a lane line image taken by a camera device on the electric bicycle; A lane line recognition unit, configured to recognize a lane line in the lane line image, and obtain first position information of the lane line on the lane line image; the lane line includes a first lane line and a second lane line; the first position information includes third position information of the first lane line on the lane line image and fourth position information of the second lane line on the lane line image; A second acquisition unit, configured to acquire second position information of a preset reference point of the lane line image based on the lane line image; the preset reference point is an image center point of the lane line image; a determination unit, configured to determine a violation of the electric bicycle based on the first location information and the second location information; The determination unit is also used to determine a first relative position relationship between the first lane line and the center point of the image based on the third position information and the second position information; if the first relative position relationship meets a first preset condition, determine a second relative position relationship between the second lane line and the center point of the image based on the fourth position information and the second position information; and determine the violation of the electric bicycle based on the second relative position relationship.

8. An electronic device, characterized in that: The electronic device comprises: one or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the method for detecting electric bicycle violations as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the method for detecting violations of electric bicycles as described in any one of claims 1 to 6.

Images