A method, device and storage medium for detecting illegal parking of shared electric motorcycles

Abstract

The present invention discloses a method, device and storage medium for detecting illegal parking of shared electric motorcycles. The present invention collects parking images of shared electric motorcycles and uses machine vision recognition technology to perform illegal parking detection of shared electric motorcycles. In this way, compared with traditional manual inspections, not only the detection workload is reduced, but also the detection efficiency is improved. At the same time, the present invention associates illegal parking detection with users of shared electric motorcycles, and generates illegal parking information and sends it to corresponding users when illegal parking occurs, thereby reminding users of the existence of illegal parking behavior. Therefore, effective supervision can be formed on users, thereby reducing illegal parking behavior of shared electric motorcycles from the source.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine vision detection, and specifically relates to a method, device and storage medium for detecting illegal parking of shared electric motorcycles. Background Art

[0002] In recent years, with the rapid development of satellite positioning systems and mobile communication technologies, shared electric vehicles have quickly become popular in society, greatly facilitating people's short-distance travel. Shared electric vehicles are a new type of transportation that can be unlocked by scanning a code and returned and shared anytime and anywhere.

[0003] However, while shared electric vehicles provide convenience to people, they also bring corresponding parking management and other problems. Among them, illegal parking of shared electric vehicles is the main difficulty in its parking management. In actual applications, shared electric vehicles are easily parked on the edge of zebra crossing areas, motor vehicle or non-motor vehicle lanes, thereby occupying the road for other vehicles, which not only easily causes road congestion, but also affects the city appearance. At present, most of them still rely on the personal qualities of users to park vehicles, and rely on operators to organize vehicles in fixed time periods to specify parking locations. However, the aforementioned parking management method has the following shortcomings: large workload, low efficiency, and inability to form an effective management method for users. Based on this, how to provide a shared electric motorcycle illegal parking detection method with low workload, high efficiency, and the ability to effectively prevent users from parking randomly has become a problem that needs to be solved urgently. Summary of the invention

[0004] The purpose of the present invention is to provide a method, device and storage medium for detecting illegal parking of shared electric motorcycles, so as to solve the problems of large workload, low efficiency and inability to form effective management means for users in the existing technology for parking management of shared electric vehicles.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, a method for detecting illegal parking of a shared electric motorcycle is provided, comprising:

[0007] Acquire a parking image of a target motorcycle, and perform target detection processing on the parking image to obtain user information of the target motorcycle;

[0008] Performing zebra crossing detection processing and lane line detection processing on the parking image to obtain a zebra crossing area and a lane line in the parking image respectively;

[0009] Based on the lane line, determining a lane area in the parking image;

[0010] Determine whether the target motorcycle is located in the zebra crossing area or in the lane area;

[0011] If yes, then based on the user information, obtain the vehicle information of the target motorcycle;

[0012] Illegal parking information is generated based on the vehicle information and the parking image, and the illegal parking information is sent to the terminal of the user of the target motorcycle, so as to complete the illegal parking detection of the target motorcycle after sending.

[0013] Based on the above disclosed content, the present invention obtains the illegal parking detection result of the target motorcycle by collecting the parking image of the target motorcycle and performing image recognition on the parking image; wherein, the specific recognition process is: first, the parking image is subjected to target detection processing to identify the user of the target motorcycle in the parking image, thereby obtaining the user information; then, the parking image is subjected to zebra crossing detection and lane line detection to obtain the zebra crossing area and lane line in the parking image; then, based on the lane line, the lane area in the parking image is extracted; in this way, the illegal parking detection result of the target motorcycle can be obtained by judging whether the target motorcycle is located in the lane area or the zebra crossing area, that is, when the target motorcycle is located in the lane area or the zebra crossing area, it is determined that the target motorcycle is illegally parked, at this time, the vehicle information can be obtained based on the user information, and the illegal parking information can be generated based on the vehicle information and the parking image and sent to the user's terminal, thereby reminding the user of the illegal parking behavior, so as to achieve the purpose of supervising the user to park the motorcycle in compliance with regulations.

[0014] Through the above design, the present invention collects parking images of shared electric motorcycles and uses machine vision recognition technology to detect illegal parking of shared electric motorcycles. In this way, compared with traditional manual inspections, not only the detection workload is reduced, but also the detection efficiency is improved. At the same time, the present invention associates illegal detection with the users of shared electric motorcycles, and when illegal parking occurs, generates illegal parking information and sends it to the corresponding users, thereby reminding the users of illegal parking behavior. As a result, effective supervision can be formed on the users, thereby reducing illegal parking behavior of shared electric motorcycles from the source.

[0015] In a possible design, performing zebra crossing detection processing on the parking image to obtain a zebra crossing area in the parking image includes:

[0016] Performing zebra crossing anchor frame detection processing on the parking image to obtain an image area corresponding to the anchor frame where the zebra crossing is located;

[0017] Performing grayscale transformation on the image area corresponding to the anchor frame where the zebra crossing is located to obtain a first grayscale image;

[0018] Extracting a zebra crossing pre-selected area from the first grayscale image, and performing a morphological opening operation on the zebra crossing pre-selected area to obtain a processed zebra crossing pre-selected area;

[0019] The processed zebra crossing pre-selected area is subjected to zebra crossing extraction processing to obtain the zebra crossing area after the zebra crossing extraction processing.

[0020] In a possible design, extracting a zebra crossing pre-selected area from the first grayscale image includes:

[0021] Performing threshold segmentation on the first grayscale image, so as to use the area in the first grayscale image whose grayscale value is greater than the grayscale threshold as the zebra crossing pre-selected area;

[0022] Correspondingly, performing zebra crossing extraction processing on the processed zebra crossing pre-selected area to obtain the zebra crossing area after the zebra crossing extraction processing includes:

[0023] Dividing the processed zebra crossing pre-selected area into a plurality of independent image blocks;

[0024] Calculate the rectangularity of each image block, and use the image block whose rectangularity is greater than the rectangularity threshold as the target image block;

[0025] Calculating the offset angle of each target image block relative to the parking image, and taking the target image block whose offset angle is less than an angle threshold as a zebra crossing image block;

[0026] Each zebra crossing image block is subjected to image merging processing, so as to obtain the zebra crossing area after the image merging processing.

[0027] In one possible design, performing lane line detection processing on the parking image to obtain lane lines in the parking image includes:

[0028] Performing grayscale transformation processing on the parking image to obtain a second grayscale image;

[0029] Performing Gaussian filtering on the second grayscale image to obtain a filtered image, and performing edge detection on the filtered image to obtain an edge detected image;

[0030] Extracting a lane line area image from the edge detection image, and performing Hough transform processing on the lane line area image to obtain a plurality of lane line pixel points after the Hough transform processing;

[0031] The plurality of lane line pixel points are subjected to fitting processing, so as to obtain lane lines in the parking image after the fitting processing.

[0032] In a possible design, extracting a lane line area image from the edge detection image includes:

[0033] A lane line mask image is acquired, and an image bitwise AND operation is performed on the lane line mask image and the edge detection image, so as to obtain the lane line area image after the image bitwise AND operation.

[0034] In a possible design, determining a lane area in the parking image based on the lane line includes:

[0035] Overlaying the lane line with the parking image to obtain the lane area after the overlay processing;

[0036] Accordingly, judging whether the target motorcycle is located in the zebra crossing area or in the lane area includes:

[0037] Performing motorcycle detection processing on the parked image to obtain the detection frame coordinates of the target motorcycle;

[0038] Determine whether the detection frame coordinates are located in the zebra crossing area or in the lane area;

[0039] If so, it is determined that the target motorcycle is located in the zebra crossing area or the lane area, and based on the user information, the vehicle information of the target motorcycle is obtained.

[0040] In a possible design, target detection processing is performed on the parking image to obtain user information of the target motorcycle, including:

[0041] Using the LBPH face recognition algorithm to perform face detection processing on the parking image, so as to obtain a face recognition result of the target motorcycle after the face detection processing;

[0042] Based on the face recognition result, face matching is performed from the motorcycle registration database to obtain the user information of the target motorcycle after face matching.

[0043] In a second aspect, a shared electric motorcycle illegal parking detection device is provided, comprising:

[0044] A target detection unit, used to obtain a parking image of a target motorcycle and perform target detection processing on the parking image to obtain user information of the target motorcycle;

[0045] A parking area detection unit, used to perform zebra crossing detection processing and lane line detection processing on the parking image, so as to obtain the zebra crossing area and lane line in the parking image respectively;

[0046] A parking area detection unit, configured to determine a lane area in the parking image based on the lane line;

[0047] An illegal parking detection unit, used to determine whether the target motorcycle is located in the zebra crossing area or in the lane area;

[0048] If so, the illegal parking detection unit obtains vehicle information of the target motorcycle based on the user information;

[0049] The illegal parking detection unit is also used to generate illegal parking information based on the vehicle information and the parking image, and send the illegal parking information to the terminal of the user of the target motorcycle, so as to complete the illegal parking detection of the target motorcycle after sending.

[0050] According to a third aspect, another device for detecting illegal parking of shared electric motorcycles is provided. Taking the device as an electronic device as an example, the device includes a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the method for detecting illegal parking of shared electric motorcycles as described in the first aspect or any one of the possible designs in the first aspect.

[0051] In a fourth aspect, a computer-readable storage medium is provided, on which instructions are stored. When the instructions are run on a computer, the method for detecting illegal parking of shared electric motorcycles as described in the first aspect or any possible design of the first aspect is executed.

[0052] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, causes the computer to execute the method for detecting illegal parking of shared electric motorcycles as described in the first aspect or any possible design of the first aspect.

[0053] Beneficial effects:

[0054] The present invention collects parking images of shared electric motorcycles and uses machine vision recognition technology to detect illegal parking of shared electric motorcycles. This not only reduces the detection workload but also improves the detection efficiency compared to traditional manual inspections. At the same time, the present invention associates illegal detection with the users of shared electric motorcycles, and when illegal parking occurs, generates illegal parking information and sends it to the corresponding users, thereby reminding the users of illegal parking. This can effectively supervise the users and reduce illegal parking of shared electric motorcycles from the source. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1A schematic diagram of the steps of a method for detecting illegal parking of a shared electric motorcycle provided by an embodiment of the present invention;

[0056] Figure 2 A schematic diagram of the effect of the zebra crossing pre-selection area provided by an embodiment of the present invention;

[0057] Figure 3 A schematic diagram of the effect of the zebra crossing pre-selected area after processing provided by an embodiment of the present invention;

[0058] Figure 4 A schematic diagram of the effect of extracting the zebra crossing area provided by an embodiment of the present invention;

[0059] Figure 5 A schematic diagram of the structure of a shared electric motorcycle illegal parking detection device provided by an embodiment of the present invention;

[0060] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0062] It should be understood that although the terms first, second, etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another unit. For example, a first unit can be referred to as a second unit, and similarly, a second unit can be referred to as a first unit without departing from the scope of the exemplary embodiments of the present invention.

[0063] It should be understood that the term "and / or" that may appear in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, B exists alone, and A and B exist at the same time. The term " / and" that may appear in this article describes another type of association object relationship, indicating that two relationships may exist. For example, A / and B can represent two situations: A exists alone, and A and B exist alone. In addition, the character " / " that may appear in this article generally indicates that the previous and next associated objects are in an "or" relationship.

[0064] Example:

[0065] See also Figures 1 to 4 As shown, the illegal parking detection method for shared electric motorcycles provided in this embodiment uses machine vision technology to detect illegal parking of shared electric motorcycles. In this way, compared with the traditional manual patrol method, the present method is more efficient and has a lower workload. At the same time, after detecting that a shared electric motorcycle is illegally parked, the present method will generate illegal parking information and send it to the user of the shared electric motorcycle to remind the user of the illegal parking behavior. Therefore, the present method can effectively supervise the user, thereby reducing the illegal parking behavior of shared electric motorcycles from the source. Based on this, the present method is suitable for large-scale application and promotion in the field of shared electric motorcycle parking management. In this embodiment, it is taken as an example that the present method can be but not limited to running on the server side. It can be understood that the aforementioned execution subject does not constitute a limitation on the embodiments of the present application. Accordingly, the operation steps of the present method can be but not limited to the following steps S1 to S6.

[0066] S1. Acquire a parking image of a target electric motorcycle, and perform target detection processing on the parking image to obtain user information of the target electric motorcycle; in this embodiment, a camera may be installed at each parking point of a shared electric motorcycle, and then the camera is used to collect parking images of the shared electric motorcycles at each parking point; at the same time, each camera transmits the collected parking images of the shared electric motorcycles to the server, and the server performs illegal parking detection of the shared electric motorcycles based on the received parking images.

[0067] Optionally, for example, but not limited to, an LBPH (local binary pattern histogram) face recognition algorithm can be used to perform face detection processing on the parked image to obtain the face recognition result of the target motorcycle after the face detection processing; wherein, the algorithm used this time is the algorithm that comes with opencv, that is, opencv provides a FaceRecognizer (face recognition) class, which contains some related face recognition algorithms and function interfaces, and this embodiment uses the LBPH FaceRecogn izer face recognition algorithm, which, with the support of opencv, can directly call the function to create a LBPH face recognition model, thereby realizing face detection in the parked image.

[0068] After obtaining the face recognition result, face matching can be performed on the motorcycle registration database to obtain the user information of the target motorcycle after face matching; specifically, the motorcycle registration database stores the face image and corresponding user information (such as name, ID number, phone number, age, etc.) of each registered user of the shared motorcycle. Therefore, after obtaining the face detection image in the parked image based on the LBPH face recognition algorithm, it is matched with the face image stored in the database to obtain the face image of the registered user that matches the face detection image. Finally, the matched registered user information is used as the user information of the target motorcycle; in this way, the association between the shared motorcycle and its user can be completed, so that after the illegal parking is detected in the subsequent period, the illegal parking information can be sent to it.

[0069] After the user of the target motorcycle is identified, illegal parking detection may be performed, wherein the detection process may be, but is not limited to, steps S2 to S5 as shown below.

[0070] S2. Perform zebra crossing detection processing and lane line detection processing on the parking image to obtain the zebra crossing area and lane line in the parking image respectively; in this embodiment, for the zebra crossing detection processing and the lane line detection processing, it is necessary to first extract the corresponding regions of interest from the parking image, and then, the zebra crossing area and lane line can be obtained based on the regions of interest of the two respectively; optionally, for example, but not limited to, the following steps S21 to S24 can be used to complete the zebra crossing detection in the parking image.

[0071] S21. Perform zebra crossing anchor frame detection on the parking image to obtain the image area corresponding to the anchor frame where the zebra crossing is located; in this embodiment, the Yolov4 model can be used as an example but is not limited to perform zebra crossing anchor frame detection, wherein the Yolov4 model consists of three parts, namely backbone, neck and head, wherein Backbone is CSPDarknet53 (CSP stands for Cross Stage Partial), which is used to extract features; the neck is composed of SPP (SpatialPyramid Pooling block) and PANet (Path Aggregation Network), which are used to increase the receptive field and separate the most important features, and PANet can ensure that semantic features are received from high-level layers and fine-grained features are received from low-level layers of the horizontal backbone network at the same time, thereby extracting richer features; finally, the head layer performs detection based on the anchor frame, and detects three feature maps of different sizes 13x13, 26x26, and 52x52 to obtain detection results.

[0072] Of course, the yolov4 model used in this embodiment is a trained model, that is, a large number of sample parking images are used as input, and the zebra crossing anchor frame in each sample parking image is used as output (referring to the coordinates) to train the model, thereby obtaining the trained model; in this way, after inputting the parking image into the trained yolov4 model, the anchor frame coordinates of the zebra crossing in the image can be obtained, and finally, according to the output anchor frame coordinates, the image area corresponding to the zebra crossing anchor frame can be obtained.

[0073] After the initial positioning of the zebra crossing area in the parking image is completed, the zebra crossing area can be extracted, wherein the extraction process can be but is not limited to the following steps S22 to S24.

[0074] S22. Perform grayscale transformation on the image area corresponding to the anchor frame where the zebra crossing is located to obtain a first grayscale image; in specific applications, the grayscale transformation can be, but is not limited to, using the cvtColor function in Opencv to convert the RGB image (i.e., the image area corresponding to the anchor frame where the zebra crossing is located) into a grayscale image; and after completing the grayscale conversion, a preselected area (i.e., the area of ​​interest) can be extracted from the zebra crossing, so that the actual zebra crossing area can be subsequently segmented from the preselected area; wherein, the extraction process of the preselected area can be, but is not limited to, as shown in the following step S23.

[0075] S23. Extract the zebra crossing pre-selected area from the first grayscale image, and perform a morphological opening operation on the zebra crossing pre-selected area to obtain the processed zebra crossing pre-selected area; in specific implementation, since the contrast between the zebra crossing and the road surface is relatively obvious, the area can be divided according to the grayscale, that is, the first grayscale image can be threshold-segmented but not limited to, so that the area with a grayscale value greater than the grayscale threshold in the first grayscale image is used as the zebra crossing pre-selected area; optionally, the grayscale threshold is set at 200, so that the area composed of pixels with a grayscale value greater than 200 can be used as the zebra crossing pre-selected area; wherein, the effect schematic diagram of the zebra crossing pre-selected area can be seen in Figure 2 shown.

[0076] After obtaining the zebra crossing pre-selected area, it is necessary to perform smoothing processing to improve the accuracy of the image. In this embodiment, morphological opening operation is used to achieve the above-mentioned effect. Specifically, image opening operation is a process in which the image is successively corroded and expanded. Small non-critical areas can be removed by corrosion, and elements that are very close can be separated. Then, the gaps left by excessive corrosion can be filled by expansion. In this way, some isolated small points can be removed by opening operation, and rough edge lines can be smoothed. At the same time, the area of ​​the original area will not change significantly, so as to achieve the effect of deburring. Optionally, the effect schematic diagram of the processed zebra crossing pre-selected area can be seen in Figure 3 shown.

[0077] In addition, before performing the aforementioned step S23, Gaussian filtering may be performed on the first grayscale image to remove noise points in the image.

[0078] After the smoothing process of the zebra crossing pre-selected area is completed, the zebra crossing can be extracted from the processed zebra crossing pre-selected area to obtain the zebra crossing area, as shown in the following step S24.

[0079] S24. Perform zebra crossing extraction processing on the processed zebra crossing pre-selected area to obtain the zebra crossing area after the zebra crossing extraction processing; in specific applications, for example, but not limited to, the following steps S24a to S24d can be used to extract the zebra crossing area from the processed zebra crossing pre-selected area.

[0080] S24a. Divide the processed zebra crossing pre-selected area into several independent image blocks; in the specific implementation, the connection operator (Regions, SeperateRegions) in the halcon operator can be used but is not limited to divide the processed zebra crossing pre-selected area into several independent image blocks, wherein Regions is the whole image, SeperateRegions is the set of broken up regions, so that the unconnected regions in Regions are regarded as independent regions, thereby obtaining multiple image blocks.

[0081] After obtaining a plurality of image blocks, it is necessary to calculate the rectangularity of each image block, so as to screen the image blocks based on the rectangularity. The rectangularity calculation process is shown in the following step S24b.

[0082] S24b. Calculate the rectangularity of each image block, and use the image block whose rectangularity is greater than the rectangularity threshold as the target image block; in this embodiment, rectangularity is an important concept in halcon, which is used to describe the shape characteristics of objects in the image, and refers to the approximation of the object's contour. A minimum circumscribed rectangle can be constructed by using the object's boundary point set, and then the object's contour is compared with the minimum circumscribed rectangle. It is one of the commonly used algorithms for shape features, and its calculation principle will not be repeated; at the same time, for example, but not limited to, the select_shape operator (SeperateRegions, SelectedRegions, 'rectangularity', 'and', 0.75, 1) in halcon can be used to filter out image blocks with a rectangularity between 0.75-1, so as to serve as target image blocks.

[0083] After the image blocks are screened, the offset angle of each screened image block relative to the parking image can be calculated, so as to screen the image blocks representing the zebra crossing based on the offset angle, wherein the calculation process of the offset angle is shown in the following step S24c.

[0084] S24c. Calculate the offset angle of each target image block relative to the parked image, and use the target image block with an offset angle less than an angle threshold as a zebra crossing image block; in this embodiment, for example, but not limited to, the halcon operator orientation_region(SelectedRegions, Ph i) is used to calculate the offset angle of all target image blocks relative to the parked image, that is, Ph i, and then, the direction of all target image blocks can be determined by the angle; finally, by setting an angle threshold, the target image blocks less than the angle threshold are used as zebra crossing image blocks; and after obtaining the zebra crossing image block, all zebra crossing image blocks can be merged to obtain the zebra crossing area, wherein the image merging process can be but not limited to as shown in the following step S24d.

[0085] S24d. Each zebra crossing image block is subjected to image merging processing to obtain the zebra crossing area after the image merging processing; in this embodiment, the effect schematic diagram of the zebra crossing area can be seen Figure 4 shown; through Figure 4 It can be seen that this embodiment can extract the zebra crossing area in the image, thereby providing a data basis for subsequent illegal parking.

[0086] Therefore, through the aforementioned steps S24a~S24d, the zebra crossing detection in the parking image can be completed, so as to extract the zebra crossing area, and then, lane line detection is required to extract the lane area in the parking image; finally, it can be determined whether the target motorcycle is located in the lane area or the zebra crossing area to obtain the detection result of its illegal parking; optionally, the following steps S24e~S24h can be used but not limited to obtain the lane lines in the parking image.

[0087] S24e. Perform grayscale transformation processing on the parking image to obtain a second grayscale image. In this embodiment, the grayscale transformation processing process of the parking image can refer to the aforementioned step S22, and its principle will not be repeated here.

[0088] After the grayscale transformation of the parking image is completed, denoising and edge detection processing may be performed, wherein the aforementioned processing process may be but is not limited to the following step S24f.

[0089] S24f. Perform Gaussian filtering on the second grayscale image to obtain a filtered image, and perform edge detection on the filtered image to obtain an edge detected image. In specific applications, for example, the GaussianBl ur function in Opencv can be used but is not limited to implement Gaussian filtering, which can remove some noise points in the original image. At the same time, edge detection can be used but is not limited to the canny edge detection algorithm. Of course, the aforementioned Gaussian filtering and canny edge algorithm are commonly used techniques for image processing, and their principles will not be repeated.

[0090] After obtaining the edge detection image, the region of interest, that is, the lane line area, can be extracted from the edge detection image so as to subsequently fit the lane line from the lane line area. The above processing process can be but is not limited to as shown in the following step S24g.

[0091] S24g. Extract a lane line area image from the edge detection image, and perform Hough transform processing on the lane line area image to obtain a number of lane line pixel points after the Hough transform processing; in specific applications, it can be but not limited to obtaining a lane line mask image, and then performing an image bitwise AND operation on the lane line mask image and the edge detection image to obtain the lane line area image after the image bitwise AND operation; wherein, the lane line is generally located in a trapezoidal area below the image, therefore, it can be but not limited to presetting 4 points to form the four vertices of the trapezoidal area, and then using the fillConvexPoly function to construct a polygon (the function has a total of 4 parameters: an empty image (size and original Figure 1 , vertex information, number of polygon edges, line color), thereby obtaining the aforementioned mask image; finally, the trapezoidal mask area is bitwise_anded with the original image to obtain only the edge detection image within the area of ​​interest, so as to obtain the lane line information from the image; in addition, in this embodiment, the Hough transform is used to detect the lane line pixels, which is a commonly used algorithm for straight line detection, and its principle will not be repeated.

[0092] After obtaining a number of lane line pixel points, a fitting process is performed on them to obtain the lane lines in the parking image, as shown in S24h after the following step.

[0093] S24h. Perform fitting processing on the plurality of lane line pixel points to obtain lane lines in the parking image after the fitting processing.

[0094] Therefore, through the aforementioned steps S24e to S24h, the lane line detection in the parking image can be completed, and then, the lane area in the parking area can be extracted based on the detected lane lines; wherein, the lane area extraction process is shown in the following step S3.

[0095] S3. Based on the lane lines, determine the lane area in the parking image; in specific applications, the lane lines are superimposed with the parking image to obtain the lane area; and after obtaining the lane area and the zebra crossing area, the illegal parking detection result can be obtained by judging whether the target motorcycle is located in the lane area or in the zebra crossing area, wherein the illegal parking detection process is shown in the following step S4.

[0096] S4. Determine whether the target motorcycle is located in the zebra crossing area or the lane area. In the present embodiment, the parking image may be first subjected to motorcycle detection processing but is not limited thereto to obtain the detection frame coordinates of the target motorcycle. Then, determine whether the detection frame coordinates are located in the zebra crossing area or the lane area. If the detection frame coordinates are located in the zebra crossing area or the lane area, it is determined that the target motorcycle has been illegally parked. In this case, it is necessary to associate the user with the vehicle and issue a reminder of the illegal parking. Furthermore, for example, the motorcycle detection processing may also use the yo lov4 model, and the aforementioned detection frame coordinates may be the coordinates of its four vertices.

[0097] Meanwhile, the reminder process of the violation may be, but is not limited to, as shown in the following steps S5 and S6.

[0098] S5. If so, then based on the user information, the vehicle information of the target motorcycle is obtained; in the present embodiment, it has been explained above that the user information includes the user's name and phone number, therefore, the order data of the user's shared motorcycle can be searched according to the user's name, that is, the latest order data can be found to obtain the vehicle information of the target motorcycle; wherein, the vehicle information can include but is not limited to the vehicle number, vehicle location, start time, parking time, etc.

[0099] After obtaining the vehicle information of the target motorcycle, it can be associated with the user to generate illegal parking information and send it to the user's corresponding terminal to remind the user of the illegal behavior; wherein, the generation process of illegal parking information can be but is not limited to as shown in the following step S6.

[0100] S6. Generate illegal parking information based on the vehicle information and the parking image, and send the illegal parking information to the terminal of the user of the target motorcycle, so as to complete the illegal parking detection of the target motorcycle after sending; in this embodiment, the parking image can be used as evidence of illegal behavior, and the generated illegal parking information can be in the form of but not limited to: Dear xx user, you used the shared motorcycle numbered xxxx from xx time to xxx time, and did not park the vehicle in accordance with regulations when parking. Please correct it in time. The illegal parking map is attached below; further, the illegal parking information of the wiring harness can be sent to the user's terminal by means of but not limited to text messages, APP push, etc., so as to remind the user of illegal parking behavior.

[0101] Thus, through the illegal parking detection method of shared electric motorcycles described in detail in the aforementioned steps S1 to S6, the present invention utilizes machine vision technology to perform illegal parking detection of shared electric motorcycles. Thus, compared with the traditional manual patrol method, the present invention is more efficient and has a lower workload. At the same time, after detecting that a shared electric motorcycle is illegally parked, the present invention will generate illegal parking information and send it to the user of the shared electric motorcycle to remind the user of the illegal parking behavior. Therefore, the present invention can effectively supervise the users, thereby reducing the illegal parking behavior of shared electric motorcycles from the source. Based on this, the present invention is suitable for large-scale application and promotion in the field of parking management of shared electric motorcycles.

[0102] like Figure 5 As shown, the second aspect of this embodiment provides a hardware device for implementing the illegal parking detection method of the shared electric motorcycle described in the first aspect of the embodiment, including:

[0103] The target detection unit is used to obtain a parking image of a target motorcycle and perform target detection processing on the parking image to obtain user information of the target motorcycle.

[0104] The parking area detection unit is used to perform zebra crossing detection processing and lane line detection processing on the parking image to respectively obtain the zebra crossing area and lane line in the parking image.

[0105] A parking area detection unit is used to determine a lane area in the parking image based on the lane line.

[0106] The illegal parking detection unit is used to determine whether the target motorcycle is located in the zebra crossing area or in the lane area.

[0107] If so, the illegal parking detection unit obtains the vehicle information of the target motorcycle based on the user information.

[0108] The illegal parking detection unit is also used to generate illegal parking information based on the vehicle information and the parking image, and send the illegal parking information to the terminal of the user of the target motorcycle, so as to complete the illegal parking detection of the target motorcycle after sending.

[0109] The working process, working details and technical effects of the device provided in this embodiment can be found in the first aspect of the embodiment and will not be described in detail here.

[0110] like Figure 6 As shown, the third aspect of this embodiment provides another device for detecting illegal parking of shared electric motorcycles. Taking the device as an electronic device as an example, it includes: a memory, a processor and a transceiver that are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the method for detecting illegal parking of shared electric motorcycles as described in the first aspect of the embodiment.

[0111] For example, the memory may include, but is not limited to, random access memory (RAM), read only memory (ROM), flash memory, first input first output (FIFO) and / or first in last out (FILO), etc. Specifically, the processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). At the same time, the processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state.

[0112] In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. For example, the processor may not be limited to a microprocessor of the STM32F105 series, a reduced instruction set computer (RISC) microprocessor, an X86 or other architecture processor, or a processor with an integrated embedded neural network processing unit (NPU); the transceiver may be, but is not limited to, a wireless fidelity (WIFI) wireless transceiver, a Bluetooth wireless transceiver, a general packet radio service technology (General Packet Radio Service, GPRS) wireless transceiver, a ZigBee protocol (a low-power LAN protocol based on the IEEE802.15.4 standard, ZigBee) wireless transceiver, a 3G transceiver, a 4G transceiver and / or a 5G transceiver, etc. In addition, the device may also include, but is not limited to, a power module, a display screen and other necessary components.

[0113] The working process, working details and technical effects of the electronic device provided in this embodiment can be found in the first aspect of the embodiment and will not be described in detail here.

[0114] The fourth aspect of this embodiment provides a computer-readable storage medium that stores instructions for the method for detecting illegal parking of shared electric motorcycles as described in the first aspect of the embodiment, that is, the computer-readable storage medium stores instructions, and when the instructions are run on a computer, the method for detecting illegal parking of shared electric motorcycles as described in the first aspect of the embodiment is executed.

[0115] The storage medium refers to a carrier for storing data, which may include but is not limited to a floppy disk, a CD, a hard disk, a flash memory, a USB flash drive and / or a memory stick, and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0116] The working process, working details and technical effects of the computer-readable storage medium provided in this embodiment can be found in the first aspect of the embodiment and will not be described in detail here.

[0117] The fifth aspect of this embodiment provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute the method for detecting illegal parking of shared electric motorcycles as described in the first aspect of the embodiment, wherein the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0118] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting illegal parking of shared electric motorcycles, characterized in that: include: Acquire a parking image of a target motorcycle, and perform target detection processing on the parking image to obtain user information of the target motorcycle; Performing zebra crossing detection processing and lane line detection processing on the parking image to obtain a zebra crossing area and a lane line in the parking image respectively; Based on the lane line, determining a lane area in the parking image; Determine whether the target motorcycle is located in the zebra crossing area or in the lane area; If yes, then based on the user information, obtain the vehicle information of the target motorcycle; Generate illegal parking information according to the vehicle information and the parking image, and send the illegal parking information to the terminal of the user of the target motorcycle, so as to complete the illegal parking detection of the target motorcycle after sending; Performing zebra crossing detection processing on the parking image to obtain a zebra crossing area in the parking image includes: Performing zebra crossing anchor frame detection processing on the parking image to obtain an image area corresponding to the anchor frame where the zebra crossing is located; Performing grayscale transformation on the image area corresponding to the anchor frame where the zebra crossing is located to obtain a first grayscale image; Extracting a zebra crossing pre-selected area from the first grayscale image, and performing a morphological opening operation on the zebra crossing pre-selected area to obtain a processed zebra crossing pre-selected area; Performing zebra crossing extraction processing on the processed zebra crossing pre-selected area to obtain the zebra crossing area after the zebra crossing extraction processing; Extracting a zebra crossing pre-selected area from the first grayscale image includes: Performing threshold segmentation on the first grayscale image, so as to use the area in the first grayscale image whose grayscale value is greater than the grayscale threshold as the zebra crossing pre-selected area; Correspondingly, performing zebra crossing extraction processing on the processed zebra crossing pre-selected area to obtain the zebra crossing area after the zebra crossing extraction processing includes: Dividing the processed zebra crossing pre-selected area into a plurality of independent image blocks; Calculate the rectangularity of each image block, and use the image block whose rectangularity is greater than the rectangularity threshold as the target image block; Calculating the offset angle of each target image block relative to the parking image, and taking the target image block whose offset angle is less than an angle threshold as a zebra crossing image block; Performing image merging processing on each zebra crossing image block, so as to obtain the zebra crossing area after the image merging processing; Performing lane line detection processing on the parking image to obtain lane lines in the parking image includes: Performing grayscale transformation processing on the parking image to obtain a second grayscale image; Performing Gaussian filtering on the second grayscale image to obtain a filtered image, and performing edge detection on the filtered image to obtain an edge detected image; Extracting a lane line area image from the edge detection image, and performing Hough transform processing on the lane line area image to obtain a plurality of lane line pixel points after the Hough transform processing; Performing fitting processing on the plurality of lane line pixel points to obtain lane lines in the parking image after the fitting processing; Extracting a lane line area image from the edge detection image includes: A lane line mask image is acquired, and an image bitwise AND operation is performed on the lane line mask image and the edge detection image, so as to obtain the lane line area image after the image bitwise AND operation.

2. The method according to claim 1, characterized in that: Determining a lane area in the parking image based on the lane line includes: Overlaying the lane line with the parking image to obtain the lane area after the overlay processing; Accordingly, judging whether the target motorcycle is located in the zebra crossing area or in the lane area includes: Performing motorcycle detection processing on the parked image to obtain the detection frame coordinates of the target motorcycle; Determine whether the detection frame coordinates are located in the zebra crossing area or in the lane area; If so, it is determined that the target motorcycle is located in the zebra crossing area or the lane area, and based on the user information, the vehicle information of the target motorcycle is obtained.

3. The method according to claim 1, characterized in that Performing target detection processing on the parking image to obtain user information of the target motorcycle includes: Using the LBPH face recognition algorithm to perform face detection processing on the parking image, so as to obtain a face recognition result of the target motorcycle after the face detection processing; Based on the face recognition result, face matching is performed from the motorcycle registration database to obtain the user information of the target motorcycle after face matching.

4. A shared electric motorcycle illegal parking detection device, characterized in that: include: A target detection unit, used to obtain a parking image of a target motorcycle and perform target detection processing on the parking image to obtain user information of the target motorcycle; A parking area detection unit, used to perform zebra crossing detection processing and lane line detection processing on the parking image, so as to obtain the zebra crossing area and lane line in the parking image respectively; A parking area detection unit, configured to determine a lane area in the parking image based on the lane line; An illegal parking detection unit, used to determine whether the target motorcycle is located in the zebra crossing area or in the lane area; If so, the illegal parking detection unit obtains vehicle information of the target motorcycle based on the user information; The illegal parking detection unit is further used to generate illegal parking information according to the vehicle information and the parking image, and send the illegal parking information to the terminal of the user of the target motorcycle, so as to complete the illegal parking detection of the target motorcycle after sending; Performing zebra crossing detection processing on the parking image to obtain a zebra crossing area in the parking image includes: Performing zebra crossing anchor frame detection processing on the parking image to obtain an image area corresponding to the anchor frame where the zebra crossing is located; Performing grayscale transformation on the image area corresponding to the anchor frame where the zebra crossing is located to obtain a first grayscale image; Extracting a zebra crossing pre-selected area from the first grayscale image, and performing a morphological opening operation on the zebra crossing pre-selected area to obtain a processed zebra crossing pre-selected area; Performing zebra crossing extraction processing on the processed zebra crossing pre-selected area to obtain the zebra crossing area after the zebra crossing extraction processing; Extracting a zebra crossing pre-selected area from the first grayscale image includes: Performing threshold segmentation on the first grayscale image, so as to use the area in the first grayscale image whose grayscale value is greater than the grayscale threshold as the zebra crossing pre-selected area; Correspondingly, performing zebra crossing extraction processing on the processed zebra crossing pre-selected area to obtain the zebra crossing area after the zebra crossing extraction processing includes: Dividing the processed zebra crossing pre-selected area into a plurality of independent image blocks; Calculate the rectangularity of each image block, and use the image block whose rectangularity is greater than the rectangularity threshold as the target image block; Calculating the offset angle of each target image block relative to the parking image, and taking the target image block whose offset angle is less than an angle threshold as a zebra crossing image block; Performing image merging processing on each zebra crossing image block, so as to obtain the zebra crossing area after the image merging processing; Performing lane line detection processing on the parking image to obtain lane lines in the parking image includes: Performing grayscale transformation processing on the parking image to obtain a second grayscale image; Performing Gaussian filtering on the second grayscale image to obtain a filtered image, and performing edge detection on the filtered image to obtain an edge detected image; Extracting a lane line area image from the edge detection image, and performing Hough transform processing on the lane line area image to obtain a plurality of lane line pixel points after the Hough transform processing; Performing fitting processing on the plurality of lane line pixel points to obtain lane lines in the parking image after the fitting processing; Extracting a lane line area image from the edge detection image includes: A lane line mask image is acquired, and an image bitwise AND operation is performed on the lane line mask image and the edge detection image, so as to obtain the lane line area image after the image bitwise AND operation.

5. A shared electric motorcycle illegal parking detection device, characterized in that: include: A memory, a processor and a transceiver which are sequentially connected in communication, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the method for detecting illegal parking of shared electric motorcycles as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on the computer, the illegal parking detection code method for shared electric motorcycles as described in any one of claims 1 to 3 is executed.

Images