Auxiliary vehicle returning method and system based on image recognition and related equipment

Abstract

The invention provides an auxiliary vehicle returning method and system based on image recognition and related equipment. The method comprises the following steps: jumping to a preset view finding interface in response to a vehicle returning operation of a user; acquiring a vehicle returning scene image based on view finding prompt information on the view finding interface; using a target detection model to identify vehicle returning element information of the vehicle returning scene image; judging whether the vehicle meets a preset vehicle returning condition or not based on the identified vehicle returning element information; when the vehicle returning element information meets the vehicle returning condition, character recognition is carried out on a license plate image through a deep learning model, and a license plate number is obtained; and sending the vehicle returning site image and the license plate number to a server and carrying out vehicle returning processing. According to the vehicle returning method, vehicle returning element information identification and vehicle returning judgment are carried out on the vehicle returning site image by using the network model, and the technical problems that positioning is easy to be inaccurate and a user cannot be supervised to carry out vehicle returning normatively in an existing vehicle returning mode based on GPS positioning are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of shared vehicles, and in particular to an auxiliary vehicle return method, system and related equipment based on image recognition. Background Art

[0002] In the management of shared transportation, users' random parking of vehicles not only affects the urban environmental order but also brings great difficulties to management. In the prior art, users mainly rely on GPS positioning to determine whether the vehicle is within the designated vehicle return area. This method in the prior art has obvious defects. On the one hand, the GPS positioning accuracy is limited, resulting in inaccurate positioning. On the other hand, this method cannot determine whether the vehicle complies with specific norms such as parking lines.

[0003] Therefore, the prior art still needs to be improved and developed. Summary of the Invention

[0004] The present invention provides an auxiliary vehicle return method, system and related equipment based on image recognition. The main purpose of the present invention is to solve the technical problems mentioned in the background art of the prior art.

[0005] The first aspect of the present invention provides an auxiliary vehicle return method based on image recognition, including:

[0006] Jumping to a preset viewfinder interface in response to a user's vehicle return operation;

[0007] Collecting an image of the vehicle return site based on the viewfinder prompt information on the viewfinder interface;

[0008] Using a target detection model to identify vehicle return element information from the image of the vehicle return site;

[0009] Judging whether the vehicle meets a preset vehicle return condition based on the identified vehicle return element information;

[0010] When the vehicle return element information meets the vehicle return condition, performing character recognition on the license plate image through a deep learning model to obtain the license plate number;

[0011] Sending the image of the vehicle return site and the license plate number to the server for vehicle return processing.

[0012] In an optional implementation manner of the first aspect of the present invention, the collecting an image of the vehicle return site based on the viewfinder prompt information on the viewfinder interface includes:

[0013] Generating a standard viewfinder frame for standardizing the composition of vehicle return elements on the viewfinder interface;

[0014] Obtaining the captured image of the camera in real time;

[0015] Adjust the framing position of the captured image by moving the camera based on the standard viewfinder;

[0016] Judge in real time whether the current state of the captured image meets the preset shooting requirements;

[0017] If the current state of the captured image meets the shooting requirements, prompt the user to take a photo.

[0018] In an optional implementation manner of the first aspect of the present invention, the judging in real time whether the current state of the captured image meets the preset shooting requirements includes:

[0019] Perform a first shooting condition judgment on whether the captured image contains necessary vehicle return elements in real time;

[0020] Perform a second shooting condition judgment on whether the image quality of the captured image meets the preset index parameters in real time;

[0021] Perform a third shooting condition judgment on whether the necessary vehicle return elements in the captured image meet the preset position distribution in real time;

[0022] When the first shooting condition, the second shooting condition, and the third shooting condition are all met, it is determined that the current state of the captured image meets the shooting requirements.

[0023] In an optional implementation manner of the first aspect of the present invention, after judging in real time whether the current state of the captured image meets the shooting requirements, it further includes:

[0024] If the current state of the captured image does not meet the shooting requirements, generate a prompt word for the unmet conditions in real time, so as to adjust the unmet part of the captured image;

[0025] After the current state of the captured image is adjusted to meet the shooting requirements, prompt the user to take a photo.

[0026] In an optional implementation manner of the first aspect of the present invention, the identifying the vehicle return element information from the vehicle return scene image by using the target detection model includes:

[0027] Use an improved target detection model based on YOLOv5 to process the vehicle return scene image, and identify vehicle information, license plate information, and stop line information in the vehicle return scene image.

[0028] In an optional implementation manner of the first aspect of the present invention, the obtaining the license plate number by performing character recognition on the license plate image through a deep learning model when the vehicle return element information meets the vehicle return conditions includes:

[0029] Extract the four-corner feature points in the license plate image using an improved Harris corner detection algorithm;

[0030] Extract the local invariant feature points in the license plate image using the SIFT algorithm;

[0031] Establish the descriptor matching between the four-corner feature points and the local invariant feature points;

[0032] Construct an affine transformation matrix based on the matching point pairs;

[0033] Use the RANSAC algorithm to eliminate the mismatched points and optimize the transformation parameters to minimize the reprojection error;

[0034] Apply the affine transformation matrix for perspective correction and use bilinear interpolation for image resampling, and then output the standardized license plate image.

[0035] In an alternative embodiment of the first aspect of the present invention, when the return vehicle element information meets the return vehicle condition, the character recognition of the license plate image by the deep learning model to obtain the license plate number further includes:

[0036] Perform adaptive binarization processing on the standardized license plate image, morphological operations to remove noise, and character segmentation based on connected component analysis;

[0037] Extract CNN features for the cut character regions. The CNN feature extraction uses a 5-layer convolutional layer structure, followed by BatchNorm and ReLU for each layer, and performs multi-scale feature fusion and attention mechanism to enhance the features of key regions;

[0038] Convert the obtained CNN feature map into a temporal feature sequence, use bidirectional GRU to process the dependencies between characters, and finally obtain the finally recognized license plate number through CTC decoding.

[0039] The second aspect of the present invention provides an auxiliary return vehicle system based on image recognition. The auxiliary return vehicle system includes:

[0040] A return vehicle response module for jumping to a preset viewfinder interface in response to the user's return vehicle operation;

[0041] An image acquisition module for acquiring the return vehicle scene image based on the viewfinder prompt information on the viewfinder interface;

[0042] An element recognition module for using a target detection model to recognize the return vehicle element information in the return vehicle scene image;

[0043] A return vehicle judgment module for judging whether the vehicle meets the preset return vehicle conditions based on the recognized return vehicle element information;

[0044] A license plate number acquisition module, configured to, when the vehicle return element information meets the vehicle return condition, perform character recognition on a license plate image through a deep learning model to obtain a license plate number;

[0045] A vehicle return processing module, configured to send the vehicle return scene image and the license plate number to a server and perform vehicle return processing.

[0046] A third aspect of the present invention provides an auxiliary vehicle return device based on image recognition. The auxiliary vehicle return device based on image recognition includes: a memory and at least one processor. Instructions are stored in the memory, and the memory and the at least one processor are interconnected through a line;

[0047] The at least one processor calls the instructions in the memory to enable the auxiliary vehicle return device based on image recognition to execute the auxiliary vehicle return method based on image recognition according to any one of the first aspects of the present invention as described above.

[0048] A fourth aspect of the present invention provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, it implements the auxiliary vehicle return method based on image recognition according to any one of the first aspects of the present invention as described above.

[0049] Beneficial effects: The present invention provides an auxiliary vehicle return method, system and related devices based on image recognition. The method includes jumping to a preset viewfinder interface in response to a user's vehicle return operation; collecting a vehicle return scene image based on the viewfinder prompt information on the viewfinder interface; using a target detection model to identify vehicle return element information in the vehicle return scene image; judging whether the vehicle meets a preset vehicle return condition based on the identified vehicle return element information; when the vehicle return element information meets the vehicle return condition, performing character recognition on a license plate image through a deep learning model to obtain a license plate number; sending the vehicle return scene image and the license plate number to a server and performing vehicle return processing. The vehicle return method of the present invention uses a network model to identify vehicle return element information in a vehicle return scene image and judge vehicle return, overcoming the technical problems of inaccurate positioning in the existing GPS-based vehicle return method and the inability to supervise users to return the vehicle in a standardized manner. Description of the Drawings

[0050] Figure 1 It is a schematic diagram of an embodiment of the main method steps of an auxiliary vehicle return method based on image recognition of the present invention;

[0051] Figure 2 It is a schematic diagram of an embodiment of the vehicle return execution logic of an auxiliary vehicle return method based on image recognition of the present invention;

[0052] Figure 3Schematic diagram of an embodiment of an auxiliary vehicle return system based on image recognition according to the present invention;

[0053] Figure 4 Schematic diagram of an embodiment of an auxiliary vehicle return device based on image recognition according to the present invention. Detailed implementation manners

[0054] The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of the present invention and the above drawings are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments described herein can be implemented in an order other than those illustrated or described herein. In addition, the term "comprising" or "having" and any variation thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily limit to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0055] For ease of understanding, the following describes the specific process of the embodiments of the present invention. Please refer to Figure 1 In the first aspect of the present invention, an auxiliary vehicle return method based on image recognition is provided. The execution subject of the auxiliary vehicle return method of the present invention includes but is not limited to a mobile terminal. The auxiliary vehicle return method includes:

[0056] S100. Jump to a preset viewfinder interface in response to the user's vehicle return operation; In this step, an exemplary scenario may be that the user clicks the vehicle return button on the vehicle return program interface of the terminal, and then the interface of the vehicle return program jumps to the preset viewfinder interface.

[0057] S200. Collect an image of the vehicle return site based on the viewfinder prompt information on the viewfinder interface; In the viewfinder interface of the vehicle return program of the present invention, a shooting standard prompt system and an image real-time prediction mechanism are set. The shooting standard prompt system mainly prompts the user on how to select the required viewfinder, and the image real-time prediction mechanism is used to detect in real time whether the image of the viewfinder contains the necessary elements for vehicle return judgment and whether the state of the image meets the requirements.

[0058] More specifically, in an optional implementation manner of step S200 of the present invention, the collecting of the image of the vehicle return site based on the viewfinder prompt information on the viewfinder interface includes:

[0059] S201. Generate a standard viewfinder on the viewfinder interface for standardizing the composition of elements for returning the vehicle. In an alternative embodiment of the present invention, the standard viewfinder includes, but is not limited to, a standard viewfinder for displaying the vehicle outline, license plate area, and stop line. In some other alternative embodiments, the standard viewfinder may also provide an angle indicator to display in real time whether the shooting angle is appropriate; display the ambient light index to prompt whether fill light is required; and provide a function for feedback on the distance between the vehicle and the stop line.

[0060] S202. Obtain the captured image of the camera in real time. In this step, the captured image of the camera is the real-time background image of the standard viewfinder. By selecting a view based on the standard viewfinder, the efficiency of subsequent image target recognition can be improved.

[0061] S203. Adjust the viewing position of the captured image by moving the camera based on the standard viewfinder. In the present invention, when taking a view with the camera, it is observed whether the required elements for returning the vehicle exist in the corresponding area of the standard viewfinder. If not, the camera can be moved or the position of the vehicle can be readjusted so that the camera can select an image that matches the standard viewfinder.

[0062] S204. Judge in real time whether the current state of the captured image meets the preset shooting requirements. In the present invention, the image captured by the camera in real time will also execute a real-time pre-judgment mechanism. The main contents included in the real-time pre-judgment mechanism are target detection pre-judgment: detecting in real time whether key elements such as vehicles, license plates, and stop lines are included in the picture; image quality pre-judgment: detecting blur, exposure, contrast, etc.; composition pre-judgment: analyzing whether the relative positions of the key elements meet the requirements. It should be noted that in the present invention, shooting is only allowed when all pre-judgment conditions are met.

[0063] Specifically, in an alternative embodiment of step S204 of the present invention, the real-time judgment of whether the current state of the captured image meets the preset shooting requirements includes: making a first shooting condition judgment on whether the captured image contains necessary elements for returning the vehicle in real time; making a second shooting condition judgment on whether the image quality of the captured image meets the preset index parameters in real time; making a third shooting condition judgment on whether the necessary elements for returning the vehicle in the captured image meet the preset position distribution in real time; when the first shooting condition, the second shooting condition, and the third shooting condition are all met, it is determined that the current state of the captured image meets the shooting requirements.

[0064] S205. If the current state of the captured image meets the shooting requirements, prompt the user to take a photo. In the present invention, a photo-taking button is provided at the edge or corner of the viewfinder interface. When the first shooting condition, the second shooting condition, and the third shooting condition in the above steps are all met, it will be prompted on the viewfinder interface that a photo can be taken.

[0065] S206. If the current state of the captured image does not meet the shooting requirements, generate a prompt word for the unmet conditions in real time, so as to adjust the part of the captured image that does not meet the conditions; after the current state of the captured image is adjusted to meet the shooting requirements, prompt the user to take a photo. In the present invention, for example, the prompt word for the unmet conditions can be that a certain element is lacking in the viewed image, the light of the viewed image is too dark and the clarity is insufficient, and so on.

[0066] S300. Use a target detection model to identify the information of the return car elements in the return car scene image; in an optional implementation manner of the first aspect of the present invention, the use of the target detection model to identify the information of the return car elements in the return car scene image includes: using an improved target detection model based on YOLOv5 to process the return car scene image, and identifying the vehicle information, license plate information, and parking line information in the return car scene image. In the present invention, depthwise separable convolution is introduced into the backbone network of the improved target detection model based on YOLOv5, and depthwise separable convolution is used to replace some standard convolutional layers. Depthwise separable convolution decomposes the standard convolution into depthwise convolution and pointwise convolution, which can significantly reduce the amount of calculation and the number of parameters, while maintaining good feature extraction ability. A bidirectional feature pyramid network (BiFPN) is introduced into the neck network of the target detection model. It not only includes a top-down path, but also adds a bottom-up path, enabling features to interact and fuse more fully between different scales. This helps the model better detect targets of different sizes, such as license plates and parking lines. In YOLOv5 of the target detection model, anchors are pre-set bounding box templates used to match the shape of the target object. The improved model can adopt an adaptive anchor mechanism to dynamically adjust the anchors according to the actual size and shape of the target object in the training data. For example, by using a clustering algorithm (such as K-means) to cluster the bounding boxes of vehicles, license plates, and parking lines in the training data, a set of anchors that more conform to the actual data distribution can be generated. This can improve the detection accuracy of the model for targets of different sizes and shapes.

[0067] S400. Determine whether the vehicle meets the preset return conditions based on the recognized vehicle return element information. In the present invention, the vehicle return element information mainly includes vehicle information (including but not limited to vehicle type, vehicle color, and vehicle attitude); license plate information (including but not limited to license plate number and license plate status); parking line information (including but not limited to parking line position, and the precise position of the parking line is determined by a parking line detection algorithm; the alignment of the vehicle with the parking line: it is judged whether the vehicle is completely parked within the parking line by the relationship between the vehicle bounding box and the parking line position).

[0068] In the present invention, for vehicle information extraction, an improved YOLOv5 model can be used to detect the vehicle and extract the bounding box and class information of the vehicle. For the vehicle color, the main color can be extracted from within the vehicle bounding box using a color histogram or a color space segmentation algorithm. For the vehicle attitude, a vehicle key point detection algorithm (such as OpenPose or other key point detection models) can be used to identify the key points of the vehicle (such as the front of the vehicle, the rear of the vehicle, etc.), so as to judge the parking attitude of the vehicle.

[0069] License plate information extraction: On the basis of vehicle detection, further detect the license plate area and extract the bounding box of the license plate. Use optical character recognition (OCR) technology to perform character recognition on the license plate area and extract the license plate number. Perform clarity detection on the license plate area, for example, judge whether the license plate is clear by calculating the gradient information of the license plate area or using a blur detection algorithm.

[0070] Parking line information extraction: An improved YOLOv5 model can be used to detect the parking line and extract the bounding box and shape information of the parking line. Calculate the alignment of the vehicle with the parking line through the geometric relationship between the vehicle bounding box and the parking line bounding box, for example, calculate the offset of the center point of the vehicle bounding box from the center point of the parking line.

[0071] In the present invention, the definition of the preset return conditions includes the following: The vehicle is completely parked within the parking line: Judge whether the vehicle bounding box is completely contained within the parking line bounding box. It can be judged by calculating the intersection over union (IoU) between the vehicle bounding box and the parking line bounding box. If the IoU is greater than a certain threshold (such as 0.9), it is considered that the vehicle is completely parked within the parking line.

[0072] The alignment accuracy of the vehicle with the parking line: Calculate the offset of the center point of the vehicle bounding box from the center point of the parking line. If the offset is less than a certain threshold (such as 10% of the parking line width), it is considered that the vehicle is well aligned with the parking line.

[0073] Vehicle stationary state: Judge whether the vehicle is completely stationary through vehicle attitude detection. For example, if the key points of the vehicle (such as the wheels) do not move significantly in several consecutive frames of images, it is considered that the vehicle has stopped.

[0074] License plate status: Determine whether the license plate is clear and unobstructed. If the clarity detection result of the license plate area indicates that the license plate is clear and the license plate area is not blocked by other objects (such as tree branches, billboards, etc.), the license plate status is considered good.

[0075] S500. When the return vehicle element information meets the return vehicle conditions, use a deep learning model to perform character recognition on the license plate image to obtain the license plate number. Specifically, in the present invention, this step includes:

[0076] (1) Use an improved Harris corner detection algorithm to extract the four-corner feature points in the license plate image; use the SIFT algorithm to extract the local invariant feature points in the license plate image; establish the descriptor matching of the four-corner feature points and the local invariant feature points; construct an affine transformation matrix based on the matching point pairs; use the RANSAC algorithm to eliminate the mismatched points and optimize the transformation parameters to minimize the reprojection error; apply the affine transformation matrix for perspective correction and use bilinear interpolation for image resampling, and then output the standardized license plate image.

[0077] (2) Perform adaptive binarization processing, morphological operations to remove noise, and character segmentation based on connected component analysis on the standardized license plate image; perform CNN feature extraction on the cut character areas. The CNN feature extraction uses a 5-layer convolutional layer structure, followed by BatchNorm and ReLU for each layer, and performs multi-scale feature fusion and attention mechanism to enhance the features of key areas; convert the obtained CNN feature map into a time series feature sequence, use a bidirectional GRU to process the dependencies between characters, and finally obtain the finally recognized license plate number through CTC decoding.

[0078] S600. Send the return vehicle scene image and the license plate number to the server and perform the return vehicle process. In the invention, after obtaining the license plate number, the system can perform the return vehicle according to the license plate number. In order to better retain the return vehicle record, at the same time, information such as the return vehicle scene image, target detection result, and return vehicle recognition result can also be saved to the server for subsequent model training and optimization iteration.

[0079] Generally speaking, the overall process logic of vehicle return in the present invention can be as Figure 2As shown in the figure, the functional modules for implementing the car return process of the present invention may include: 1. Image acquisition module: Collect images of the car return site through the camera of the mobile device. 2. Target detection module: Use an improved target detection model based on YOLOv5 to process the collected images, and identify target objects such as vehicles, license plates, and stop lines and their position information. 3. License plate alignment module: If the car return conditions are met, further locate and align the detected license plate area, and use the affine transformation algorithm based on OpenCV to correct the license plate image to eliminate the effects of inclination, rotation, etc., and obtain a normalized license plate image. 4. Character recognition module: Use a deep learning model based on CNN and GRU to perform character recognition on the aligned license plate image to obtain the license plate number. 5. Result processing module: Return the recognized license plate number result to the user, and at the same time save information such as the original image, detection result, and recognition result to the server for subsequent model training and optimization iteration.

[0080] Furthermore, in order to better improve the processing speed of the car return method of the present invention, the software system corresponding to the method of the present invention can adopt a distributed multi-threaded parallel processing architecture, which can efficiently process a large number of user requests simultaneously, improve the throughput and response speed of the overall system. At the same time, the system has good scalability and can be horizontally extended according to actual needs to support a larger number of requests. In this way, the above-mentioned modules are connected to each other through an efficient data pipeline to achieve an end-to-end automated recognition and judgment process. At the same time, the system background integrates an automatic annotation and model training module, which can continuously optimize and improve the recognition accuracy of the model.

[0081] Some of the main features used in the technical solution of the present invention include: providing an innovative two-stage license plate recognition method, combining the high-performance target detection model of YOLOv5 and the license plate character recognition model based on CNN and GRU, which can accurately and efficiently detect and recognize license plate information; providing a license plate image alignment technology based on affine transformation, which can effectively eliminate the geometric deformation of the license plate area in the image, such as rotation and inclination; a distributed multi-threaded parallel processing architecture, which achieves high throughput through multi-threaded concurrent processing and can efficiently respond to a large number of user requests simultaneously. etc., thereby greatly improving the accuracy of subsequent character recognition. An automatic annotation and model training module is provided, which can continuously optimize and iterate model parameters, and continuously improve the accuracy and robustness of license plate recognition. A mobile terminal-based image acquisition method is provided, which does not require the deployment of additional hardware, reducing the system cost and usage threshold.

[0082] Generally speaking, the technical solution of the present invention is based on image recognition and uses a neural network model to judge vehicle return. Compared with traditional algorithms, it has stronger robustness and applicability, can efficiently and accurately identify parking lines, vehicle positions, and license plate information, greatly improve the accuracy of vehicle return judgment, and reduce manual misjudgment. Moreover, it also has a high degree of automation: the entire recognition and judgment process does not require manual intervention and can operate autonomously and continuously 7*24 hours, greatly reducing labor costs. Fast recognition speed: It adopts a multi-threaded design and can process multiple requests in parallel. Fast response speed: The invention adopts a distributed multi-threaded parallel processing architecture and can efficiently process a large number of user requests simultaneously. High recognition accuracy: On the publicly available license plate dataset, the accuracy of the license plate character recognition model of the invention is higher than that of traditional algorithms. Broad application prospects: The present invention can not only be applied to the vehicle return management of shared transportation tools, but also be extended to multiple fields such as license plate recognition in parking lots, traffic violation investigation, and vehicle security monitoring.

[0083] See Figure 3 , the second aspect of the present invention provides an auxiliary vehicle return system based on image recognition, and the auxiliary vehicle return system includes:

[0084] A vehicle return response module 10, configured to jump to a preset viewfinder interface in response to a user's vehicle return operation;

[0085] An image acquisition module 20, configured to acquire a vehicle return scene image based on the viewfinder prompt information on the viewfinder interface;

[0086] An element recognition module 30, configured to use a target detection model to recognize vehicle return element information from the vehicle return scene image;

[0087] A vehicle return judgment module 40, configured to judge whether the vehicle meets a preset vehicle return condition based on the recognized vehicle return element information;

[0088] A number acquisition module 50, configured to perform character recognition on the license plate image through a deep learning model to obtain the license plate number when the vehicle return element information meets the vehicle return condition;

[0089] A vehicle return processing module 60, configured to send the vehicle return scene image and the license plate number to the server and perform vehicle return processing.

[0090] In an optional implementation manner of the second aspect of the present invention, the image acquisition module includes:

[0091] A viewfinder frame generation unit, configured to generate a standard viewfinder frame for standardizing the composition of vehicle return elements on the viewfinder interface;

[0092] An image real-time acquisition unit, configured to acquire the captured image of the camera in real time;

[0093] A viewfinder adjustment unit for adjusting the viewfinder position of the camera for the captured image based on the standard viewfinder frame;

[0094] A shooting judgment unit for judging in real time whether the current state of the captured image meets the preset shooting requirements;

[0095] A first photographing unit for prompting the user to take a photo if the current state of the captured image meets the shooting requirements.

[0096] In an optional implementation manner of the second aspect of the present invention, the shooting judgment unit includes:

[0097] A first shooting condition judgment subunit for performing a first shooting condition judgment on whether the captured image contains necessary vehicle-return elements in real time;

[0098] A second shooting condition judgment subunit for performing a second shooting condition judgment on whether the image quality of the captured image meets the preset index parameters in real time;

[0099] A third shooting condition judgment subunit for performing a third shooting condition judgment on whether the necessary vehicle-return elements in the captured image meet the preset position distribution in real time;

[0100] A determination subunit for determining that the current state of the captured image meets the shooting requirements when the first shooting condition, the second shooting condition, and the third shooting condition are all satisfied.

[0101] In an optional implementation manner of the second aspect of the present invention, the image acquisition module further includes:

[0102] An adjustment prompt unit for generating a prompt word for unmet conditions in real time if the current state of the captured image does not meet the shooting requirements, so as to adjust the unmet part of the captured image;

[0103] A second photographing unit for prompting the user to take a photo after the current state of the captured image is adjusted to meet the shooting requirements.

[0104] In an optional implementation manner of the second aspect of the present invention, the element recognition module includes:

[0105] A target detection unit for processing the vehicle-return site image using an improved target detection model based on YOLOv5 to identify vehicle information, license plate information, and parking line information in the vehicle-return site image.

[0106] In an optional implementation manner of the second aspect of the present invention, the number acquisition module includes:

[0107] The first feature point extraction unit is used to extract the four-corner feature points in the license plate image by using an improved Harris corner detection algorithm;

[0108] The second feature point extraction unit is used to extract local invariant feature points in the license plate image by using the SIFT algorithm;

[0109] The matching establishment unit is used to establish the descriptor matching between the four-corner feature points and the local invariant feature points;

[0110] The matrix construction unit is used to construct an affine transformation matrix based on the matching points;

[0111] The error elimination unit is used to eliminate the mismatched points by using the RANSAC algorithm, optimize the transformation parameters, and minimize the reprojection error;

[0112] The image correction output unit is used to perform perspective correction by applying the affine transformation matrix, and perform image resampling by using bilinear interpolation, and then output the standardized license plate image.

[0113] In an optional implementation manner of the second aspect of the present invention, the number acquisition module further includes:

[0114] The pixel-level processing unit is used to perform adaptive binarization processing on the standardized license plate image, perform morphological operations to remove noise, and perform character segmentation based on connected component analysis;

[0115] The CNN feature extraction unit is used to perform CNN feature extraction on the cut character regions. The CNN feature extraction uses a 5-layer convolutional layer structure, followed by BatchNorm and ReLU for each layer, and performs multi-scale feature fusion and attention mechanism to enhance the features of key regions;

[0116] The GRU sequence modeling unit is used to convert the obtained CNN feature map into a temporal feature sequence, utilize bidirectional GRU to process the dependencies between characters, and finally obtain the finally recognized license plate number through CTC decoding.

[0117] Figure 4FIG. 0 is a schematic structural diagram of an image recognition-based auxiliary car return device provided by an embodiment of the present invention. The image recognition-based auxiliary car return device may vary greatly due to different configurations or performances, and may include one or more processors 70 (central processing units, CPUs) (for example, one or more processors) and a memory 80, and one or more storage media 90 for storing application programs or data (for example, one or more mass storage devices). Among them, the memory and the storage medium may be transient storage or persistent storage. The program stored in the storage medium may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the image recognition-based auxiliary car return device. Further, the processor may be configured to communicate with the storage medium and execute a series of instruction operations in the storage medium on the image recognition-based auxiliary car return device.

[0118] The image recognition-based auxiliary car return device of the present invention may further include one or more power supplies 100, one or more wired or wireless network interfaces 110, one or more input / output interfaces 120, and / or one or more operating systems, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art can understand that Figure 4 The shown structural diagram of the image recognition-based auxiliary car return device does not constitute a limitation on the image recognition-based auxiliary car return device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0119] The present invention also provides a computer-readable storage medium. The computer-readable storage medium may be a non-volatile computer-readable storage medium, or may also be a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions run on a computer, the computer is caused to execute the steps of the above-mentioned image recognition-based auxiliary car return method.

[0120] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described system or systems, units can refer to the corresponding processes in the foregoing method embodiments, and will not be described herein again.

[0121] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0122] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An auxiliary vehicle return method based on image recognition, characterized in that: include: In response to the user's vehicle return operation, jumping to a preset viewfinder interface; Collecting images of the vehicle return scene based on the framing prompt information on the framing interface; Using a target detection model to identify vehicle return element information on the vehicle return scene image; Determining whether the vehicle meets the preset return conditions based on the identified vehicle return element information; When the vehicle return element information satisfies the vehicle return condition, character recognition is performed on the license plate image through a deep learning model to obtain the license plate number; The vehicle return scene image and the license plate number are sent to a server and the vehicle return process is performed.

2. The image recognition-based assisted vehicle return method according to claim 1, characterized in that: The collecting of the vehicle return scene image based on the framing prompt information on the framing interface includes: Generating a standard framing frame for standardizing the composition of the vehicle return elements on the framing interface; Get the captured image of the camera in real time; Moving the camera based on the standard framing frame to adjust the framing position of the captured image; Determine in real time whether the current state of the captured image meets the preset shooting requirements; If the current state of the captured image meets the shooting requirement, the user is prompted to take a photo.

3. The image recognition-based assisted vehicle return method according to claim 2, characterized in that: The real-time determination of whether the current state of the acquired image meets the preset shooting requirements includes: Performing a first shooting condition judgment on whether the captured image contains necessary elements for returning the vehicle in real time; Determine in real time whether the image quality of the captured image meets a second shooting condition of a preset index parameter; Determining in real time whether the necessary vehicle return elements in the captured image meet a third shooting condition of a preset position distribution; When the first shooting condition, the second shooting condition and the third shooting condition are all satisfied, it is determined that the current state of the acquired image satisfies the shooting requirement.

4. The image recognition-based assisted vehicle return method according to claim 2, characterized in that: After the real-time determination of whether the current state of the acquired image meets the shooting requirements, the method further includes: If the current state of the collected image does not meet the shooting requirements, a prompt word indicating that the conditions are not met is generated in real time, so as to adjust the part of the collected image that does not meet the conditions; When the current state of the captured image is adjusted to meet the shooting requirement, the user is prompted to take a photo.

5. The image recognition-based assisted vehicle return method according to claim 1, characterized in that: The identifying of the vehicle return element information of the vehicle return scene image using the target detection model includes: The vehicle return scene image is processed using an improved target detection model based on YOLOv5 to identify vehicle information, license plate information and parking line information in the vehicle return scene image.

6. The image recognition-based assisted vehicle return method according to claim 1, characterized in that: When the vehicle return element information satisfies the vehicle return condition, character recognition is performed on the license plate image through a deep learning model to obtain the license plate number, including: Using an improved Harris corner detection algorithm to extract four corner feature points in the license plate image; Using SIFT algorithm to extract local invariant feature points in the license plate image; Establishing descriptor matching between the four corner feature points and the local invariant feature points; Construct an affine transformation matrix based on matching point pairs; Use the RANSAC algorithm to remove mismatched points and optimize the transformation parameters to minimize the reprojection error; The affine transformation matrix is ​​applied to perform perspective correction, and bilinear interpolation is used to perform image resampling, and then a standardized license plate image is output.

7. The image recognition-based assisted vehicle return method according to claim 6, characterized in that: When the vehicle return element information satisfies the vehicle return condition, performing character recognition on the license plate image through a deep learning model to obtain the license plate number further includes: Adaptively binarizing the standardized license plate image, removing noise through morphological operations, and performing character segmentation based on connected domain analysis; CNN feature extraction is performed on the cut character area. The CNN feature extraction uses a 5-layer convolution layer structure, each layer is followed by BatchNorm and ReLU, and multi-scale feature fusion and attention mechanism are performed to enhance the key area features; The obtained CNN feature map is converted into a temporal feature sequence, and the dependencies between characters are processed using a bidirectional GRU. Finally, the final recognized license plate number is obtained through CTC decoding.

8. An auxiliary vehicle return system based on image recognition, characterized in that: The auxiliary vehicle return system comprises: A vehicle return response module, configured to jump to a preset viewfinder interface in response to a vehicle return operation by a user; An image acquisition module, used to acquire images of the vehicle return scene based on the framing prompt information on the framing interface; An element recognition module, used to use a target detection model to recognize the element information of the vehicle return on-site image; A vehicle return judgment module, used to judge whether the vehicle meets the preset vehicle return conditions based on the identified vehicle return element information; A number acquisition module, used to perform character recognition on the license plate image through a deep learning model to obtain the license plate number when the return element information meets the return condition; The vehicle return processing module is used to send the vehicle return scene image and the license plate number to the server and perform vehicle return processing.

9. An auxiliary vehicle return device based on image recognition, characterized in that: The image recognition-based auxiliary vehicle return device includes: a memory and at least one processor, the memory stores instructions, and the memory and the at least one processor are interconnected via a line; The at least one processor calls the instructions in the memory to enable the image recognition-based assisted vehicle return device to execute the image recognition-based assisted vehicle return method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image recognition-based assisted vehicle return method as described in any one of claims 1 to 7 is implemented.

Images