Parking space state identification method and device, electronic equipment and storage medium

Abstract

The invention discloses a parking space state identification method and device, electronic equipment and a storage medium, and the method comprises the steps: controlling the rotation and zooming of a camera according to the pre-stored camera rotation angle information and camera zooming information corresponding to a parking space in a monitoring region for the parking space corresponding to each spatial position in a patrol inspection period, a first image corresponding to the parking space is collected; for each parking space, inputting the first image corresponding to the parking space into a pre-trained vehicle detection model, and determining a parking space state of the parking space based on the vehicle detection model; wherein the probability value of a vehicle parked in the parking space is determined based on the vehicle detection model; if the probability value is greater than a preset probability threshold value, determining that the parking space state is that a vehicle is parked, otherwise, determining that the parking space state is that no vehicle is parked; and updating the spatial position number and the parking space state corresponding to each parking space to a parking space chain table. And the timeliness and the accuracy of parking space state identification are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a parking space state recognition method and device, electronic equipment and a storage medium. BACKGROUND

[0002] Under the background of "smart city" and "safe city", the topics of "smart parking" and "convenient parking" have become important indicators of the happiness index of residents' daily travel life. In order to better meet the demand for vehicle parking when people travel by car, urban multi-level parking spaces are increasingly favored by people because they provide more vehicle parking spaces in limited urban space. Therefore, as the first step of the "smart parking" business, the parking space state recognition accuracy is a key indicator of the parking management scheme, and its advantages and disadvantages will directly affect people's travel experience. The current parking area still uses manual inspection to monitor and identify the vehicle parking posture and parking space occupation state. This scheme requires a large amount of manpower, and the timeliness of manual inspection for parking space recognition is poor. Furthermore, the parking spaces of multi-level parking spaces will move, and manual inspection cannot accurately determine the parking space state of each space position, resulting in poor accuracy of parking space recognition. SUMMARY

[0003] The present application provides a parking space state recognition method, device, electronic equipment and storage medium to solve the problem of large manpower cost, poor timeliness and accuracy of the prior art parking space state recognition.

[0004] In a first aspect, the present application provides a parking space state recognition method, which comprises:

[0005] In a patrol cycle, for each parking space corresponding to a space position in a monitoring area, the camera rotation angle information and the camera zoom information corresponding to the parking space are pre-saved, the camera rotation and zoom are controlled according to the camera rotation angle information and the camera zoom information, and the first image corresponding to the parking space is collected;

[0006] For each parking space, the first image corresponding to the parking space is input into a pre-trained vehicle detection model, and the parking space state of the parking space is determined based on the vehicle detection model. Wherein, the probability value of the parking space having a vehicle is determined based on the vehicle detection model; if the probability value is greater than a preset probability threshold, it is determined that the parking space state is that a vehicle is parked, otherwise it is determined that the parking space state is that no vehicle is parked;

[0007] The space position number and the parking space state corresponding to each parking space are updated to the parking space linked list.

[0008] The above technical solution has the following advantages or beneficial effects:

[0009] In the present application, the camera rotation angle information and the camera zoom information corresponding to each parking space in the monitoring area are pre-stored, so as to realize accurate shooting of the first image corresponding to the parking space. Then, the parking space state of the parking space is determined based on the trained vehicle detection model. And the space position number and the parking space state corresponding to each parking space are updated to the parking space linked list. Compared with the parking space state recognition scheme relying on artificial inspection, the timeliness of the parking space state recognition is improved, and the labor cost is reduced. Moreover, the recognized parking space state and the space position number of the parking space are updated to the parking space linked list, so as to realize timely determination of the parking space state of each space position, and improve the accuracy of the parking space state recognition.

[0010] In a second aspect, the present application provides a parking space state recognition device, the device comprising:

[0011] An image acquisition module is configured to, in a patrol cycle, for each parking space corresponding to a space position in a monitoring area, control the camera rotation and zoom according to the pre-stored camera rotation angle information and camera zoom information corresponding to the parking space, and acquire the first image corresponding to the parking space.

[0012] A parking space state recognition module is configured to, for each parking space, input the first image corresponding to the parking space into a pre-trained vehicle detection model, and determine the parking space state of the parking space based on the vehicle detection model. Wherein, the probability value of the parking space having a vehicle is determined based on the vehicle detection model. If the probability value is greater than a preset probability threshold, it is determined that the parking space state is having a vehicle, otherwise it is determined that the parking space state is not having a vehicle.

[0013] An updating module is configured to update the space position number and the parking space state corresponding to each parking space to a parking space linked list.

[0014] In a third aspect, the present application provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus.

[0015] The memory is configured to store a computer program.

[0016] The processor is configured to execute the program stored on the memory, and implement the method.

[0017] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method.

[0018] In a fifth aspect, the present application provides a computer program product, which comprises an executable program, and the executable program is executed by a processor to realize the method. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0020] Figure 1 A parking space state recognition process schematic diagram is provided for the present application.

[0021] Figure 2 A stereo garage inspection monitoring scene schematic diagram is provided for the present application.

[0022] Figure 3 A parking space linked list construction schematic diagram is provided for the present application.

[0023] Figure 4 A stereo parking space camera inspection schematic diagram of m*n architecture is provided for the present application.

[0024] Figure 5 A parked vehicle target detection and recognition schematic diagram is provided for the present application.

[0025] Figure 6 A picture adjustment direction schematic diagram is provided for the present application.

[0026] Figure 7 A parking space state recognition flowchart is provided for the present application.

[0027] Figure 8 A parking space state recognition device structure schematic diagram is provided for the present application.

[0028] Figure 9 An electronic device structure schematic diagram is provided for the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose and embodiments of the present application more clear, the following will combine the drawings in the exemplary embodiments of the present application to clearly and completely describe the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only some embodiments of the present application, but not all embodiments.

[0030] It should be noted that the brief description of the terms in the present application is only for the convenience of understanding the following described embodiments, and is not intended to limit the embodiments of the present application. Unless otherwise specified, these terms should be understood according to their ordinary and general meanings.

[0031] The terms "first", "second", "third", and the like in the description and in the claims of the present application and above figures are used for distinguishing between similar or identical objects and do not necessarily have a specific temporal or chronological order, unless otherwise noted. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

[0032] The terms "comprise", "comprising", "include", "including", "have", "having", "contain", "containing", "characterized by" and the like are used in the detailed description and in the claims of this application to specify the presence of stated features, integers, steps or components but do not preclude the presence or addition of one or more other features, integers, steps, components or groups thereof.

[0033] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software codes that can perform the functions related to the element.

[0034] It should be noted that the above-mentioned embodiments are merely used to illustrate the technical solutions of the present application, but not limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: they can still make modifications to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to part or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0035] For the convenience of explanation, the above description has been made in combination with specific embodiments. However, the above exemplary discussion is not intended to exhaust or limit the embodiments to the specific forms disclosed above. Various modifications and variations can be derived according to the above teachings. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different modified embodiments suitable for specific use considerations.

[0036] Figure 1 The parking space state recognition process provided in the present application includes the following steps:

[0037] S101: In the inspection cycle, for each parking space corresponding to a space position in the monitoring area, according to the pre-saved camera rotation angle information and camera zoom information corresponding to the parking space, control the camera rotation and zoom, and collect the first image corresponding to the parking space;

[0038] S102: For each parking space, input a first image corresponding to the parking space into a pre-trained vehicle detection model, and determine the parking space status of the parking space based on the vehicle detection model; wherein, a probability value of a vehicle being parked in the parking space is determined based on the vehicle detection model; if the probability value is greater than a preset probability threshold, the parking space status is determined to be parked; otherwise, the parking space status is determined to be free of parked vehicles;

[0039] S103: updating the space position number and parking space status corresponding to each parking space into the parking space linked list.

[0040] The parking space status recognition method provided in this application is applied to electronic devices, which may be PCs, computers, servers and other devices.

[0041] In the parking space status recognition scenario, a camera is deployed that is fixed in position and can rotate horizontally and vertically. This parking space status recognition scenario includes parking space status recognition scenarios in a multi-story parking garage. By rotating horizontally and / or vertically, the camera can obtain the first image corresponding to each parking space in the monitoring area. For example, in a three-story parking space scenario, the camera can monitor the first images of parking spaces corresponding to 3×3=9 spatial locations. Each parking space can move left or right or up or down, but the spatial position numbering is fixed. For example, the spatial positions on the top floor are numbered 1, 2, and 3 from left to right; the spatial positions on the second floor are numbered 4, 5, and 6 from left to right; and the spatial positions on the bottom floor are numbered 7, 8, and 9 from left to right. Vehicles enter and exit the parking spaces from the bottom floor, and each parking space can move left or right or up or down. A camera inspection route is pre-planned. For example, the inspection route is spatially numbered 1, 2, 3, 4, 5, 6, 7, 8, and 9; or the inspection route is spatially numbered 9, 8, 7, 6, 5, 4, 3, 2, and 1, etc. At each parking space capture location, the camera will stay for a preset time, zooming in and out within the preset time, and completing the parking space status identification. The preset time length is, for example, 1000ms or 2000ms. During each inspection cycle, the parking space status of all parking spaces in the monitoring area must be identified according to the pre-planned inspection route and the preset stop points.

[0042] In the present application, in a patrol cycle, for each parking space corresponding to a spatial position in the monitoring area, the camera rotation angle information and the camera zoom information corresponding to the parking space are pre-stored, the camera rotation and zoom are controlled, and the first image corresponding to the parking space is collected. It should be noted that the motor of the camera is controlled by a pulse signal to realize the horizontal rotation or vertical rotation of the camera. The correspondence between each patrol position and the corresponding pulse signal is pre-stored. According to the above correspondence, the camera is controlled to the corresponding patrol position by the corresponding pulse signal, and the zoom information corresponding to each patrol position is pre-stored. After reaching the patrol position, zooming is performed, and then the first image of the parking space corresponding to the patrol position is collected.

[0043] For each parking space, the first image corresponding to the parking space is input into a pre-trained vehicle detection model, and the probability value of a vehicle stopping in the parking space is determined based on the vehicle detection model. If the probability value is greater than a preset probability threshold, it is determined that the parking space is in a state of stopping a vehicle, otherwise it is determined that the parking space is in a state of not stopping a vehicle. The preset probability threshold is, for example, 0.85, 0.9, etc. The training process of the vehicle detection model is as follows: the parking space images and the corresponding parking state labels in the training set are input into the vehicle detection model to be trained, the vehicle detection prediction result is output based on the vehicle detection model to be trained, and the loss value is determined according to the prediction result and the parking state label. Through iterative training, when the number of training reaches a threshold or the loss value meets the requirements, the vehicle detection model training is completed.

[0044] After the parking space state of each parking space in the monitoring area is identified, the spatial position number and the parking space state corresponding to each parking space are updated to the parking space chain table. The parking lot monitoring personnel can know the parking space with a vehicle, the parking space without a vehicle, and the spatial position information of the parking space with a vehicle and the spatial position information of the parking space without a vehicle according to the information in the parking space chain table.

[0045] In the present application, for each parking space corresponding to a spatial position in the monitoring area, the camera rotation angle information and the camera zoom information corresponding to the parking space are pre-stored, thereby realizing accurate shooting of the first image corresponding to the parking space. Then, the parking space state of the parking space is determined based on the trained vehicle detection model. And the spatial position number and the parking space state corresponding to each parking space are updated to the parking space chain table. Compared with the parking space state identification scheme relying on manual patrol, the timeliness of parking space state identification is improved, and the labor cost is reduced. Moreover, the identified parking space state and the spatial position number of the parking space are updated to the parking space chain table. Thus, the parking space state of each spatial position is determined in time, and the accuracy of parking space state identification is improved.

[0046] In an optional embodiment, if the parking space state of the parking space is determined to be stopping a vehicle based on the vehicle detection model, the method further comprises:

[0047] determining a first vehicle region sub-image in the first image based on the vehicle detection model;

[0048] inputting the first vehicle region sub-image into a pre-trained license plate recognition model, and determining license plate information of the vehicle based on the license plate recognition model;

[0049] updating the space position number and the parking space state corresponding to each parking space in the parking space linked list, including:

[0050] updating the space position number, the parking space state, and the license plate information of the vehicle in the parking space state in the parking space linked list.

[0051] In the present application, if the parking space state of the parking space is determined to be parked based on the vehicle detection model, the vehicle detection model will output the first vehicle region sub-image in the first image where the vehicle is located while determining the probability value of the vehicle in the parking space.

[0052] inputting the first vehicle region sub-image into a pre-trained license plate recognition model, and determining license plate information of the vehicle in the parking space based on the license plate recognition model. The training process of the license plate recognition model is as follows: input the license plate region image and the corresponding license plate information label in the training set into the license plate recognition model to be trained, output the predicted license plate information based on the license plate recognition model to be trained, and determine the loss value according to the predicted license plate information and the license plate information label. Through iterative training, when the number of training reaches a threshold or the loss value meets the requirements, the training of the license plate recognition model is completed.

[0053] updating the space position number, the parking space state, and the license plate information of the vehicle in the parking space state in the parking space linked list. The parking lot monitoring personnel can know the parking space of the vehicle in the parking lot, the parking space without the vehicle, and the space position information of the parking space with the vehicle, the space position information of the parking space without the vehicle, and the license plate information of the vehicle in the parking space with the vehicle based on the information in the parking space linked list. The information in the parking space linked list is more complete, which facilitates the management of the parking lot and the vehicle in the parking lot by the parking lot monitoring personnel.

[0054] In an optional embodiment, after determining the first vehicle region sub-image in the first image based on the vehicle detection model, before inputting the first vehicle region sub-image into a pre-trained license plate recognition model, the method further includes:

[0055] determining a Euclidean distance between the first center point of the first image and a second center point of the first vehicle region sub-image, and if the Euclidean distance is less than a preset distance threshold, inputting the first vehicle region sub-image into the pre-trained license plate recognition model.

[0056] In the present application, according to the coordinate information of the first center point of the first image in the first image and the coordinate information of the second center point of the first vehicle region sub-image in the first image, the Euclidean distance between the first center point of the first image and the second center point of the first vehicle region sub-image can be determined. If the Euclidean distance is less than the preset distance threshold, it indicates that the first vehicle region sub-image is closer to the center of the camera's field of view, and at this time it can be considered that the distortion of the first vehicle region sub-image is smaller, and then the step of inputting the first vehicle region sub-image into the pre-trained license plate recognition model is performed, and the license plate information is determined based on the license plate recognition model, which can ensure the accuracy of the recognized license plate information.

[0057] If the Euclidean distance is not less than the preset distance threshold, before inputting the first vehicle region sub-image into the pre-trained license plate recognition model, the method further comprises:

[0058] obtaining length information and width information of the first image, and determining offset position information according to the length information, width information and position information of the first vehicle region sub-image;

[0059] adjusting the camera rotation angle information corresponding to the parking space according to the offset position information;

[0060] controlling the camera rotation and zooming according to the adjusted camera rotation angle information corresponding to the parking space and the pre-stored camera zooming information, and collecting an adjusted second image corresponding to the parking space; inputting the second image into the vehicle detection model, determining a second vehicle region sub-image in the second image based on the vehicle detection model; taking the second vehicle region sub-image as a first vehicle region sub-image, and performing the step of inputting the first vehicle region sub-image into the pre-trained license plate recognition model.

[0061] If the Euclidean distance between the first center point of the first image and the second center point of the first vehicle region sub-image is not less than the preset distance threshold, it indicates that the first vehicle region sub-image is far from the center of the camera's field of view, and at this time it is considered that the distortion of the first vehicle region sub-image may affect the accuracy of license plate recognition. Based on the above consideration, in order to improve the accuracy of license plate recognition, the length information and width information of the first image are obtained, and the offset position information is determined according to the length information, width information and position information of the first vehicle region sub-image.

[0062] wherein,

[0063] wherein the first vehicle region sub-image has an upper left corner coordinate (x1, y1) and a lower right corner coordinate (x2, y2), the first image has length information and width information (w, h), and the offset position information is (Δx, Δy).

[0064] After the offset position information is determined, the camera rotation angle information corresponding to the parking space is adjusted according to the offset position information. The target of the adjustment is that the center point of the image collected by the camera coincides with the second center point of the first vehicle region sub-image. The adjustment of the camera rotation angle information includes adjustment of horizontal rotation angle information and / or adjustment of vertical rotation angle information. According to the offset position information and the pre-labeled image movement position information corresponding to one degree of camera rotation, the horizontal rotation angle information and / or the vertical rotation angle information that the camera needs to adjust can be determined. The camera is controlled to rotate according to the horizontal rotation angle information and / or the vertical rotation angle information that needs to be adjusted through a pulse signal.

[0065] After the camera rotation is completed, the camera zoom is controlled according to the zoom information of the camera corresponding to the currently inspected parking space, and then the adjusted second image corresponding to the parking space is collected. Then, the second image is input into the vehicle detection model, and the second vehicle region sub-image in the second image is determined based on the vehicle detection model. At this time, the second vehicle region sub-image is closer to the center of the field of view of the camera, and at this time, it can be considered that the distortion of the second vehicle region sub-image is smaller. The second vehicle region sub-image is taken as the first vehicle region sub-image, and the step of inputting the first vehicle region sub-image into the pre-trained license plate recognition model is performed. Based on the license plate recognition model, the license plate information is determined, which can ensure the accuracy of the recognized license plate information.

[0066] In an optional embodiment, inputting the first vehicle region sub-image into the pre-trained license plate recognition model and determining the license plate information of the vehicle based on the license plate recognition model include:

[0067] Based on the vehicle detection model, the first vehicle region sub-image in each frame of parking space image acquired within a preset time length is determined, and based on the license plate recognition model, the candidate license plate information of the vehicle in each first vehicle region sub-image is determined. The candidate license plate information that appears the most times is determined as the license plate information of the vehicle.

[0068] The preset time length is, for example, 1000 ms, 2000 ms, etc. The camera can collect multiple frames of parking space images in the preset time length, determine a first vehicle region sub-image in each frame of the parking space images obtained in the preset time length based on the vehicle detection model, and determine candidate license plate information of the vehicle in each first vehicle region sub-image based on the license plate recognition model. For example, the camera collects 10 frames of parking space images in the preset time length, so that 10 pieces of candidate license plate information of the vehicle can be obtained. Then, the candidate license plate information with the highest occurrence frequency in the 10 pieces of candidate license plate information is determined as the license plate information of the vehicle. In this way, the accuracy of license plate recognition is further improved.

[0069] In an optional implementation, before the step of, in the inspection period, for each parking space corresponding to a spatial position in the monitoring area, controlling the camera to rotate and zoom according to the pre-stored camera rotation angle information and camera zoom information corresponding to the parking space, and collecting a first image corresponding to the parking space, the method further includes:

[0070] Based on the ultrasonic sensor installed at each parking space corresponding to a spatial position in the monitoring area, when it is determined that a vehicle enters or leaves the parking space, the subsequent step of, in the inspection period, for each parking space corresponding to a spatial position in the monitoring area, controlling the camera to rotate and zoom according to the pre-stored camera rotation angle information and camera zoom information corresponding to the parking space, and collecting a first image corresponding to the parking space is performed.

[0071] In the stereo garage scene, the parking state of each space position is likely to change only when a vehicle enters or leaves the position. If no vehicle enters or leaves the position, the parking state of each space position is generally unchanged. Therefore, in order to save the energy consumption of parking state identification, the application installs an ultrasonic sensor at each space position. Generally, the ultrasonic sensor is installed at the bottom of the parking space. The ultrasonic sensor can identify whether a vehicle enters or leaves the position by measuring the distance. For example, when the time from emission to reception of the reflected ultrasonic wave is greater than the time threshold, the ultrasonic sensor state value is 0; when the time from emission to reception of the reflected ultrasonic wave is not greater than the time threshold, the ultrasonic sensor state value is 1. When the ultrasonic sensor state value jumps from 0 to 1, it indicates that a vehicle enters the position; when the state value jumps from 1 to 0, it indicates that a vehicle leaves the position. If it is determined based on the ultrasonic sensor installed at each space position in the monitoring area that no vehicle enters or leaves the position, the subsequent step of controlling the camera to rotate and zoom according to the pre-stored camera rotation angle information and camera zoom information of the corresponding camera of the parking space, and collecting the first image corresponding to the parking space is not performed for each space position in the monitoring area in the inspection period. The garage manager can manage the garage according to the latest parking position linked list. If it is determined based on the ultrasonic sensor installed at each space position in the monitoring area that a vehicle enters or leaves the position, the subsequent step of controlling the camera to rotate and zoom according to the pre-stored camera rotation angle information and camera zoom information of the corresponding camera of the parking space, and collecting the first image corresponding to the parking space is performed for each space position in the monitoring area in the inspection period. At this time, the parking position linked list is updated, thereby ensuring the real-time and accuracy of the parking position linked list data. The accuracy of the garage manager managing the garage according to the updated parking position linked list is ensured.

[0072] In an alternative embodiment, the method further comprises:

[0073] If the parking state of each parking space in the monitoring area has been identified, the camera is controlled to reset to the initial inspection position in the preset inspection path.

[0074] For example, there are 9 parking spaces in the monitoring area, and in each inspection process, the inspection starts from the initial inspection position in the preset inspection path. The parking state of the 9 parking spaces is inspected in this way. After all the parking spaces are identified, the camera is controlled to reset to the initial inspection position in the preset inspection path. That is, the camera is controlled to rotate horizontally and vertically to the initial inspection position in the preset inspection path. In this way, the efficiency of parking state identification can be improved in the next inspection.

[0075] The vehicle state recognition process provided by the application is described below in combination with a scene schematic diagram.

[0076] The application is based on a three-dimensional vehicle parking scene, and a rotating camera such as a ball camera is used to capture and recognize vehicles in the monitoring area in a patrol mode, and the license plate recognition result pR is associated with the parking space pN corresponding to the space position number in the parking list parking_list, so as to realize autonomous patrol and recognition of the vehicle parking state in the three-dimensional parking space. First, according to the three-dimensional parking space structure and the number of parking spaces, the rotating camera routing path routing_path encoding and the parking list parking_list construction are respectively completed; wherein the parking list hanging node information parking_node includes three parts of parking space number pN, parking space state pStatus and license plate recognition result pR. Then, the direction of the state change of the parking space ultrasonic sensor is monitored to realize the vehicle entry / exit state recognition. If there is a vehicle entry / exit, the ee_flag state flag bit is set, and the state value of ee_flag is used to drive the camera to patrol the parking space. When the camera patrols the parking space, the vehicle parking state is detected and recognized one by one according to the established encoding path. This method mainly uses a deep learning network model to detect and recognize the current encoding position parking space pC. If the parking space pC has a vehicle, the parking space state pStatus is set to 1, otherwise it is 0, and the current parking space number pC, parking space state pStatus and license plate recognition result pR are hung under the corresponding parking_node in parking_list, to complete the parking space association. Finally, when the patrol list routing_list is empty, the node information results of each parking space in the patrol parking list in the current monitoring area are updated and reported, and the patrol camera is reset to the starting position node. The main execution process of the application is as follows:

[0077] Figure 2 A three-dimensional garage patrol monitoring scene schematic diagram is provided for the application. It includes the parking state recognition of 9 parking spaces. Figure 3 A parking list construction schematic diagram is provided for the application.

[0078] Parking list construction: according to the structure of the three-dimensional garage in the monitoring and patrol area and the number of vehicle parking spaces, the parking list parking_list is constructed, that is, assuming that the structure of the three-dimensional garage is m*n, then the maximum length L of the parking list parking_list is m*n. The parking list hanging node information parking_node includes three parts of parking space number pN, parking space state pStatus and license plate recognition result pR. The parking space number, parking space state and target category in parking_node are all initialized to -1, and the license plate recognition is initialized to 0.

[0079] Inspection path encoding: according to the physical space structure of the stereo parking space, the camera inspection path is specified by demonstration. The camera inspection displacement is controlled by a stepping motor or a servo motor, etc. The rotation angle of the motor is mainly controlled by inputting a specified pulse signal, and then the target moving direction and moving distance are controlled. Therefore, the purpose of encoding the inspection path routing_path can be achieved by storing the pulse signal of the motor demonstration driving in the pulse signal list pulse_list. The analog signal is converted into a digital signal for camera driving. Figure 4 The stereo parking space camera inspection schematic diagram provided in the present application is of m*n architecture.

[0080] Inspection parameter configuration: preset lens magnification len_gain and parking space state recognition parking time Δt. When the camera is inspected, the optimal viewing angle for detecting and recognizing the target of the parked vehicle in the current parking space can be obtained by magnification, and then the detection and recognition of the parked vehicle in the parking space are completed within the recognition parking time Δt, which is 1000 ms by default. Figure 5 The parked vehicle target detection and recognition schematic diagram provided in the present application.

[0081] Entry / exit recognition: the entry / exit state of the current parking space vehicle is identified by the jump direction of the parking space ultrasonic sensor state value. That is, when the sensor state value jumps from 0 to 1, it indicates that the vehicle is in place; when the state value jumps from 1 to 0, it indicates that the vehicle is out of place. If the sensor state value jumps, the entry / exit flag ee_flag is set to 1 (the default value is 0).

[0082] Parking space inspection: load the encoded inspection path routing_list and the preset lens magnification len_gain, set the inspection start flag routing_start to 0, and complete the parking space inspection preset parameter initialization work. The camera moves from the default initial position to the inspection starting position, and the monitoring field of view also switches from the panoramic image to the single parking space inspection monitoring state.

[0083] Vehicle detection: a target detection method based on deep learning is adopted, a network detection model is built, and a model training is performed on the data set of the stereo garage scene. The target vehicle on the camera inspection parking space is detected and positioned by the trained detection model, the detection frame BBox and the category attribute are obtained, and the camera field of view is adjusted by the deviation control vector v_c=[△x,△y] so that the target vehicle is located in the center of the picture. The deviation control vector is composed of the center point C1 of the vehicle detection frame and the current picture center point C2. Figure 6 The picture adjustment direction schematic diagram provided in the present application.

[0084] Assuming the target vehicle detection box BBox has left-top corner coordinates (x1, y1) and right-bottom corner coordinates (x2, y2), the width and height of the camera monitoring picture are (w, h), and the deviation control vector v_c is calculated as shown in the formula:

[0085]

[0086] License plate recognition: using a deep learning framework to train a license plate recognition model, taking the vehicle target detection box BBox on the parking space as the input of the license plate recognition model, performing license plate detection and recognition, and through the fusion of multiple frame image license plate recognition results within the parking time △t, the final recognition result pR of the target vehicle is optimized. If no license plate recognition result is obtained, the vehicle target on the parking space is recorded as a vehicle without a license plate.

[0087] Target snapshot: within the camera inspection parking time △t, if a vehicle target is detected on the parking space, the camera will trigger a snapshot, taking the ROI image of the vehicle detection result as the snapshot image reported by the system, and setting the current parking space status pStatus to 1, otherwise setting it to 0.

[0088] Parking space association: the vehicle tray in the stereo garage is not fixed, and the entry or exit of the vehicle will cause the spatial position of the vehicle tray to change. Therefore, when the camera inspects the parking space one by one, the recognition result of the current parking space must be associated with the corresponding node information in the parking space linked list parking_list. This can monitor the spatial position of the target vehicle in the stereo garage in real time, facilitating subsequent vehicle exit scheduling operations.

[0089] Parking space linked list update: after the camera inspection is completed, the parking space linked list data that needs to be uploaded to the platform is updated, and the inspection start flag routing_start is set to 0. At this time, the parking space linked list has recorded the latest spatial position of the current parked vehicle in the stereo garage, the license plate recognition result, and the vehicle snapshot image.

[0090] Camera reset: the camera inspects each parking space according to the established inspection path, and when the parking space inspection path node list routing_list is empty, the camera returns to the initial position of the full-view monitoring of the monitoring area, and sets the entry / exit flag ee_flag to 0.

[0091] Result reporting: the event reporting update of the stereo garage inspection system takes the parking space list parking_list as the information carrier, updates and reports the latest parking space status inspection results of each parking space node in parking_list, and the reporting content covers the front license plate number pN, the parking space status pStatus, the license plate recognition result pR, and the snapshot image of the corresponding vehicle.

[0092] Figure 7The parking space state recognition flowchart provided in the application includes: parking space linked list construction, inspection path coding, inspection parameter configuration, parking sensor monitoring, entry / exit space recognition, judging whether ee_flag is 0, if yes (Y), continuing parking sensor monitoring, if no (N), parking inspection, vehicle detection, license plate recognition, target snapshot, parking association, judging whether Routing_list is 0, if no (N), continuing parking inspection, if yes (Y), parking linked list updating, camera resetting, result reporting.

[0093] 1. In the stereo garage parking space scene, the physical space position of the vehicle parking space is mapped and modeled to construct the inspection parking space linked list parking_list, and the state value change of the parking ultrasonic sensor flag state state_flag is used to drive the rotating camera such as the ball camera to capture and recognize the license plate of the vehicle in the parking space node in the inspection linked list parking_list in the inspection mode, so as to realize the autonomous inspection and recognition of the vehicle parking space state in the stereo parking space. Compared with the scheme of relying on manual inspection or ground buried geomagnetic sensor or other microwave detection device, the scheme of visual detection and parking ultrasonic sensor integration is more convenient and reliable, and has better anti-interference ability.

[0094] 2. Each parking space is inspected by a pre-set magnification, which can capture higher quality view pictures as input images for vehicle detection, effectively improving the vehicle detection accuracy; and by using the deviation control vector v_c=[△x,△y], the camera view is adjusted so that the target vehicle is located in the center of the picture, which is more beneficial to the detection and recognition of the license plate target.

[0095] 3. The monitoring camera inspection mode is triggered by the state change of the parking board ultrasonic sensor, the state of each parking space in the inspection path and the corresponding vehicle license plate recognition result are updated to the corresponding parking node parking_node in the parking_list according to the camera inspection path coding value, the real-time monitoring of the spatial position of the target vehicle in the stereo garage is realized, and the subsequent vehicle dispatching operation is facilitated.

[0096] The application provides a visual detection and ultrasonic sensor fusion scheme, which captures and recognizes vehicles in a monitoring area in a patrol mode through a sensor trigger ball machine or the like rotating camera, and maps and associates parking lot license plate recognition results pR and parking lots pN of corresponding numbers in a parking lot linked list according to a patrol path coding value, so as to realize autonomous panoramic patrol recognition of vehicle parking states in a three-dimensional parking lot. Each parking lot is patrolled and associated in a preset magnification and patrol path routing_list coding mode, a better vehicle detection image is captured, and the spatial position of a target vehicle in the three-dimensional garage is monitored in real time; a camera view is adaptively adjusted through a deviation control vector v_c=[△x,△y], and a vehicle detection result BBox under the view is taken as an input of a license plate detection and recognition module.

[0097] Figure 8 A parking space state recognition device structure schematic diagram provided by the application includes:

[0098] An image acquisition module 201 is configured to, in a patrol period, control a camera rotation and zooming according to pre-stored camera rotation angle information and camera zooming information corresponding to each parking space in a monitoring area, and acquire a first image corresponding to the parking space.

[0099] A parking space state recognition module 202 is configured to input the first image corresponding to each parking space into a pre-trained vehicle detection model, and determine a parking space state of the parking space based on the vehicle detection model; wherein a probability value of a vehicle parked in the parking space is determined based on the vehicle detection model; if the probability value is greater than a preset probability threshold, it is determined that the parking space state is that a vehicle is parked, otherwise it is determined that the parking space state is that a vehicle is not parked.

[0100] An updating module 203 is configured to update a space position number and a parking space state corresponding to each parking space to a parking lot linked list.

[0101] The parking space state recognition module 202 is further configured to determine a first vehicle region sub-image in the first image based on the vehicle detection model, input the first vehicle region sub-image into a pre-trained license plate recognition model, and determine license plate information of the vehicle based on the license plate recognition model.

[0102] The updating module 203 is specifically configured to update the space position number, the parking space state and the license plate information of the vehicle parked in the parking space corresponding to each parking space to the parking lot linked list.

[0103] The parking space status recognition module 202 is also used to determine the Euclidean distance between the first center point of the first image and the second center point of the first vehicle area sub-image. If the Euclidean distance is less than a preset distance threshold, the step of inputting the first vehicle area sub-image into a pre-trained license plate recognition model is performed.

[0104] The parking space status recognition module 202 is also used to obtain the length information and width information of the first image, and determine the offset position information based on the length information, width information and the position information of the first vehicle area sub-image; adjust the camera rotation angle information corresponding to the parking space according to the offset position information; control the camera rotation and magnification according to the adjusted camera rotation angle information corresponding to the parking space and the pre-saved camera magnification information, and collect the adjusted second image corresponding to the parking space; input the second image into the vehicle detection model, and determine the second vehicle area sub-image in the second image based on the vehicle detection model; use the second vehicle area sub-image as the first vehicle area sub-image, and perform the step of inputting the first vehicle area sub-image into the pre-trained license plate recognition model.

[0105] The parking space status recognition module 202 is specifically used to determine the first vehicle area sub-image in each frame of the parking space image acquired within a preset time length based on the vehicle detection model, and determine the candidate license plate information of the vehicle in the first vehicle area sub-image in each frame based on the license plate recognition model; and determine the candidate license plate information that appears the most times as the license plate information of the vehicle.

[0106] The device further comprises:

[0107] The judgment module 204 is configured to trigger the image acquisition module 201 when judging that a vehicle enters or leaves a parking space based on the ultrasonic sensors installed at the parking spaces corresponding to each spatial position in the monitoring area.

[0108] The device further comprises:

[0109] The reset module 205 is configured to control the camera to reset to an initial inspection position in a preset inspection path if the parking status of each parking space in the monitoring area has been identified.

[0110] The present application also provides an electronic device, such as Figure 9 As shown, it includes: a processor 301, a communication interface 302, a memory 303 and a communication bus 304, wherein the processor 301, the communication interface 302, and the memory 303 communicate with each other through the communication bus 304;

[0111] The memory 303 stores a computer program which, when executed by the processor 301, causes the processor 301 to perform any of the above method steps.

[0112] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0113] The communication interface 302 is used for communication between the above electronic device and other devices.

[0114] The memory can include a Random Access Memory (RAM) and can also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0115] The processor mentioned above can be a general-purpose processor, including a central processing unit, a network processor (NP), and the like; can also be a Digital Signal Processing (DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and the like.

[0116] The present application also provides a computer storage readable storage medium, which stores a computer program executable by an electronic device, and when the program runs on the electronic device, causes the electronic device to perform any of the above method steps.

[0117] The present application provides a computer program product, which includes an executable program which, when executed by a processor, implements the method.

[0118] Although the preferred embodiments of the present application have been described, those skilled in the art can make further changes and modifications to the embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including all changes and modifications falling within the scope of the present application.

[0119] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A parking space status recognition method, characterized in that: The method comprises: During the inspection cycle, for each parking space corresponding to a spatial position in the monitoring area, the camera rotation and zoom are controlled according to pre-stored camera rotation angle information and camera zoom information corresponding to the parking space, and a first image corresponding to the parking space is captured; For each parking space, a first image corresponding to the parking space is input into a pre-trained vehicle detection model, and the parking space status of the parking space is determined based on the vehicle detection model; wherein a probability value of a vehicle being parked in the parking space is determined based on the vehicle detection model; if the probability value is greater than a preset probability threshold, the parking space status is determined to be a vehicle parked; otherwise, the parking space status is determined to be a vehicle not parked; The space position number and parking space status corresponding to each parking space are updated in the parking space linked list.

2. The method according to claim 1, wherein If it is determined based on the vehicle detection model that the parking space status is that a vehicle is parked, the method further includes: determining a first vehicle region sub-image in the first image based on the vehicle detection model; Inputting the first vehicle region sub-image into a pre-trained license plate recognition model, and determining the license plate information of the vehicle based on the license plate recognition model; Updating the space position number and parking status corresponding to each parking space into the parking space linked list includes: The spatial position number, parking space status and license plate information of the vehicle parked in the parking space corresponding to each parking space are updated in the parking space linked list.

3. The method according to claim 2, wherein After determining a first vehicle region sub-image in the first image based on the vehicle detection model and before inputting the first vehicle region sub-image into a pre-trained license plate recognition model, the method further includes: Determine the Euclidean distance between the first center point of the first image and the second center point of the first vehicle area sub-image. If the Euclidean distance is less than a preset distance threshold, input the first vehicle area sub-image into a pre-trained license plate recognition model.

4. The method according to claim 3, wherein If the Euclidean distance is not less than a preset distance threshold, before inputting the first vehicle region sub-image into a pre-trained license plate recognition model, the method further includes: Acquire length information and width information of the first image, and determine offset position information based on the length information, width information, and position information of the first vehicle region sub-image; Adjusting the camera rotation angle information corresponding to the parking space according to the offset position information; According to the adjusted camera rotation angle information corresponding to the parking space and the pre-saved camera magnification information, the camera rotation and magnification are controlled, and an adjusted second image corresponding to the parking space is collected; the second image is input into the vehicle detection model, and a second vehicle area sub-image in the second image is determined based on the vehicle detection model; the second vehicle area sub-image is used as the first vehicle area sub-image, and the first vehicle area sub-image is input into a pre-trained license plate recognition model.

5. The method according to claim 2, wherein Inputting the first vehicle region sub-image into a pre-trained license plate recognition model, and determining the license plate information of the vehicle based on the license plate recognition model includes: Based on the vehicle detection model, the first vehicle area sub-image in each frame of parking space image acquired within a preset time length is determined; based on the license plate recognition model, the candidate license plate information of the vehicle in the first vehicle area sub-image in each frame is determined; and the candidate license plate information that appears the most times is determined as the license plate information of the vehicle.

6. The method according to claim 1, wherein During the inspection cycle, for each parking space corresponding to a spatial position within the monitoring area, controlling the rotation and zoom of the camera according to pre-stored camera rotation angle information and camera zoom information corresponding to the parking space, and before capturing a first image corresponding to the parking space, the method further includes: Based on the ultrasonic sensors installed at the parking spaces corresponding to each spatial position in the monitoring area, when it is determined that a vehicle enters or leaves a parking space, a subsequent inspection cycle is performed, for each parking space corresponding to the spatial position in the monitoring area, according to the pre-saved camera rotation angle information and camera magnification information corresponding to the parking space, the camera rotation and magnification are controlled, and a first image corresponding to the parking space is captured.

7. The method according to claim 1, wherein The method further comprises: If the parking space status of each parking space in the monitoring area has been identified, the camera is controlled to reset to an initial inspection position in a preset inspection path.

8. A parking space status recognition device, characterized in that: The device comprises: An image acquisition module is configured to control the rotation and zoom of the camera for each parking space corresponding to a spatial position within the monitoring area during an inspection cycle, based on pre-stored camera rotation angle information and camera zoom information corresponding to the parking space, and to acquire a first image corresponding to the parking space; a parking space status recognition module, configured to input, for each parking space, a first image corresponding to the parking space into a pre-trained vehicle detection model, and determine the parking space status of the parking space based on the vehicle detection model; wherein, based on the vehicle detection model, a probability value of a vehicle being parked in the parking space is determined; if the probability value is greater than a preset probability threshold, the parking space status is determined to be parked; otherwise, the parking space status is determined to be free of parked vehicles; The updating module is used to update the space position number and parking space status corresponding to each parking space into the parking space linked list.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; A processor, configured to implement the method according to any one of claims 1 to 7 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images