A vehicle parking space relevance determination method, device, equipment and medium

Abstract

Embodiments of the present application provide a vehicle parking space relevance determination method, device, equipment and medium to solve the problem of inaccurate relevance of vehicles and parking spaces determined in the prior art. In the embodiments of the present application, the electronic device detects the vehicle regions of each vehicle in the to-be-processed image, and determines a vehicle labeled image corresponding to the target vehicle. The vehicle labeled image is an image in which the to-be-processed image has added label information of the vehicle region of the target vehicle. The electronic device performs key point detection on the vehicle labeled image, determines vehicle key points and parking space key points in the vehicle region of the target vehicle, and determines the relevance of the parking space associated with the target vehicle and the parking space key points based on the determined vehicle key points and parking space key points. Since the vehicle contains multiple vehicle key points and the parking space contains multiple parking space key points, even if some parts of the vehicle or the parking space are blocked, the relevance of the vehicle and the parking space can still be accurately determined.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, device and medium for determining the correlation of vehicle parking spaces. Background Technology

[0002] In static traffic scenarios, vehicles are crucial detection targets, and the correlation between vehicles and parking spaces is often a key detection objective. This correlation refers to which parking space a vehicle is parked in, or the probability of it being parked there. By analyzing this correlation, parking space management can be effectively implemented, such as controlling whether a vehicle is parked outside a designated space. Specifically, after detecting the correlation, the equipment can use methods like projection to determine if the vehicle is accurately parked within the designated space. Therefore, determining the correlation between vehicles and parking spaces is of paramount importance.

[0003] However, existing technologies for determining the correlation between vehicles and parking spaces typically involve identifying the wheel positions of vehicles in captured images and then determining the correlation between the vehicle and the parking space based on the wheel positions and the parking space lines. However, the environment around parking spaces is often complex, with many vehicles and pedestrians, so vehicle wheels are frequently obscured by other vehicles or pedestrians. Therefore, identifying the wheel positions from the captured images has a significant error, leading to inaccurate correlation determinations based on the wheel positions and parking space lines. Summary of the Invention

[0004] This application provides a method, apparatus, device, and medium for determining the correlation between vehicles and parking spaces, in order to solve the problem of inaccurate correlation between vehicles and parking spaces determined in the prior art.

[0005] In a first aspect, embodiments of this application provide a method for determining the correlation of vehicle parking spaces, the method comprising:

[0006] Detect vehicle regions of each vehicle in the image to be processed; determine the vehicle annotation image corresponding to the target vehicle; wherein, the vehicle annotation image is an image in the image to be processed with annotation information of the vehicle region of the target vehicle added, and the target vehicle is the vehicle in the image to be processed;

[0007] Key point detection is performed on the vehicle-annotated image to determine the vehicle key points and parking space key points in the vehicle region of the target vehicle.

[0008] Based on the vehicle key points and the parking space key points, the correlation between the target vehicle and the parking space is determined; wherein, the parking space is the parking space associated with the parking space key points.

[0009] Further, the step of performing key point detection on the vehicle-annotated image to determine vehicle key points and parking space key points in the vehicle region of the target vehicle includes:

[0010] The vehicle-labeled image is input into a pre-trained recognition model to obtain the vehicle key points and parking space key points in the vehicle region of the target vehicle output by the recognition model.

[0011] Furthermore, determining the correlation between the target vehicle and the parking space based on the vehicle key points and the parking space key points includes:

[0012] Based on the location information of the parking space key points in the image to be processed, determine the parking space associated with the parking space key points;

[0013] The target vehicle is determined to be a vehicle associated with the parking space.

[0014] Further, the step of inputting the labeled vehicle image into a pre-trained recognition model to obtain vehicle key points and parking space key points in the vehicle region of the target vehicle output by the recognition model includes:

[0015] The vehicle-labeled image is input into a pre-trained recognition model to obtain the vehicle key points and parking space key points in the vehicle region of the target vehicle output by the recognition model, as well as the offsets corresponding to the vehicle key points and parking space key points; wherein, the offsets are the offsets of the vehicle key points and parking space key points relative to the center point of their respective vehicle regions.

[0016] Furthermore, after determining the vehicle key points and parking space key points in the vehicle area of ​​the target vehicle, and before determining the correlation between the target vehicle and the parking space based on the vehicle key points and the parking space key points, the method further includes:

[0017] For each vehicle key point and parking space key point, based on the position information of the key point in the image to be processed and the offset of the key point relative to the center point of its corresponding vehicle area, the position information of the center point of the vehicle area corresponding to the key point in the image to be processed is determined. It is then determined whether the position information is located in the area where the center point of the target vehicle's vehicle area is located. If not, the key point is deleted.

[0018] Furthermore, after detecting the vehicle regions of each vehicle in the image to be processed and before determining the vehicle annotation image corresponding to the target vehicle, the method further includes:

[0019] The vehicle area of ​​each detected vehicle is magnified by a preset factor;

[0020] For the magnified vehicle area, perform the subsequent step of determining the vehicle annotation image corresponding to the target vehicle.

[0021] Furthermore, the vehicle regions of each vehicle in the image to be processed include:

[0022] The image to be processed is input into a pre-trained vehicle recognition model to obtain the vehicle regions of each vehicle in the image to be processed, as output by the vehicle recognition model.

[0023] Secondly, embodiments of this application also provide a device for determining the correlation of vehicle parking spaces, the device comprising:

[0024] The detection and determination module is used to detect the vehicle regions of each vehicle in the image to be processed; determine the vehicle annotation image corresponding to the target vehicle; wherein, the vehicle annotation image is an image in the image to be processed with annotation information of the vehicle region of the target vehicle added, and the target vehicle is the vehicle in the image to be processed; perform key point detection on the vehicle annotation image to determine the vehicle key points and parking space key points in the vehicle region of the target vehicle.

[0025] The processing module is used to determine the correlation between the target vehicle and the parking space based on the vehicle key points and the parking space key points; wherein the parking space is the parking space associated with the parking space key points.

[0026] Furthermore, the detection and determination module is specifically used to input the vehicle-labeled image into a pre-trained recognition model to obtain the vehicle key points and parking space key points in the vehicle region of the target vehicle output by the recognition model.

[0027] Furthermore, the processing module is specifically used to determine the parking space associated with the parking space key point based on the position information of the parking space key point in the image to be processed; and to determine that the target vehicle is a vehicle associated with the parking space.

[0028] Furthermore, the detection and determination module is specifically used to input the vehicle labeled image into a pre-trained recognition model to obtain the vehicle key points and parking space key points in the vehicle region of the target vehicle output by the recognition model, as well as the offsets corresponding to the vehicle key points and parking space key points; wherein, the offsets are the offsets of the vehicle key points and parking space key points relative to the center point of their respective vehicle regions.

[0029] Furthermore, the processing module is also used to, for each vehicle key point and parking space key point, determine the position information of the center point of the vehicle area corresponding to the key point in the image to be processed based on the position information of the key point in the image to be processed and the offset of the key point relative to the center point of its corresponding vehicle area, determine whether the position information is located in the area where the center point of the vehicle area of ​​the target vehicle is located, and if not, delete the key point.

[0030] Furthermore, the detection and determination module is also used to magnify the vehicle area of ​​each detected vehicle by a preset factor; and for the magnified vehicle area, to perform the subsequent step of determining the vehicle annotation image corresponding to the target vehicle.

[0031] Furthermore, the detection and determination module is specifically used to input the image to be processed into a pre-trained vehicle recognition model to obtain the vehicle regions of each vehicle in the image to be processed output by the vehicle recognition model.

[0032] Thirdly, embodiments of this application also provide an electronic device, the electronic device including a processor, the processor being configured to execute a computer program stored in a memory to implement the steps of determining the correlation of vehicle parking spaces as described above.

[0033] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the vehicle parking space correlation determination method as described above.

[0034] In this embodiment of the application, the electronic device detects the vehicle regions of each vehicle in the image to be processed; obtains the vehicle annotation image corresponding to the target vehicle, wherein the vehicle annotation image is an image in the image to be processed with annotation information of the vehicle region of the target vehicle added, the target vehicle is the vehicle in the image to be processed, the electronic device performs key point detection on the vehicle annotation image, determines the vehicle key points and parking space key points in the vehicle region of the target vehicle, and determines the correlation between the target vehicle and the parking space based on the determined vehicle key points and parking space key points, wherein the parking space is the parking space associated with the vehicle key points. In this embodiment, the electronic device detects the vehicle regions of each vehicle in the image to be processed and determines the vehicle annotation image corresponding to the target vehicle. The vehicle annotation image is an image in the image to be processed with annotation information of the vehicle region of the target vehicle added. The electronic device performs key point detection on the vehicle annotation image to determine the vehicle key points and parking space key points in the vehicle region of the target vehicle. Based on the determined vehicle key points and parking space key points, the correlation between the target vehicle and the parking space associated with the parking space key points is determined. Since a vehicle contains multiple vehicle key points and a parking space contains multiple parking space key points, even if some parts of the vehicle or parking space are occluded, the correlation between the vehicle and the parking space can still be accurately determined. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 A schematic diagram illustrating the process of determining the correlation of a vehicle parking space, as provided in an embodiment of this application;

[0037] Figure 2 A schematic diagram of an image to be processed provided in an embodiment of this application;

[0038] Figure 3 A schematic diagram of the center point of the vehicle area corresponding to each determined key point, provided as an embodiment of this application;

[0039] Figure 4 A schematic diagram of key points of a vehicle and key points of a parking space provided in an embodiment of this application;

[0040] Figure 5 A schematic diagram illustrating the processing of a recognition model provided in an embodiment of this application;

[0041] Figure 6 A schematic diagram of a vehicle parking space correlation determination device provided in an embodiment of this application;

[0042] Figure 7 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation

[0043] The present application will now be described in further detail with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present application.

[0044] To improve the accuracy of determining the correlation between vehicles and parking spaces, embodiments of this application provide a method, apparatus, device, and medium for determining the correlation between vehicles and parking spaces.

[0045] The method for determining the correlation between a vehicle and a parking space includes: in this embodiment, an electronic device detects the vehicle regions of each vehicle in the image to be processed; obtains a vehicle annotation image corresponding to the target vehicle, wherein the vehicle annotation image is an image in the image to be processed with annotation information of the vehicle region of the target vehicle added, the target vehicle is the vehicle in the image to be processed, the electronic device performs key point detection on the vehicle annotation image, determines the vehicle key points and parking space key points in the vehicle region of the target vehicle, and determines the correlation between the target vehicle and the parking space based on the determined vehicle key points and parking space key points, wherein the parking space is the parking space associated with the vehicle key points.

[0046] Example 1:

[0047] Figure 1 This application provides a schematic diagram of a process for determining the correlation of a vehicle parking space, which includes the following steps:

[0048] S101: Detect the vehicle regions of each vehicle in the image to be processed; determine the vehicle annotation image corresponding to the target vehicle; wherein, the vehicle annotation image is an image in the image to be processed with annotation information of the vehicle region of the target vehicle added, and the target vehicle is the vehicle in the image to be processed.

[0049] The vehicle parking space correlation determination method provided in this application embodiment is applied to an electronic device, which can be a data acquisition device or a server or other intelligent device.

[0050] In this embodiment of the application, the electronic device can acquire the image to be processed acquired by the acquisition device. Specifically, if the electronic device is an acquisition device, after acquiring the image to be processed, the acquisition device can acquire the image to be processed acquired by itself. If the electronic device is a server or other non-acquisition device intelligent device, after acquiring the image to be processed, the acquisition device can send the acquired image to the server or other non-acquisition device intelligent device, and the server or other non-acquisition device intelligent device can acquire the image to be processed acquired by the acquisition device.

[0051] After acquiring the image to be processed from the acquisition device, the electronic device can identify the vehicle region of each vehicle in the acquired image. The vehicle region can be a detection box that completely contains the vehicle, or it can be the position information of the detection box that completely contains the vehicle. The position information of the detection box can be the position information of the upper left and lower right corners of the detection box (the upper, lower, left, and right described here refer to the upper, lower, left, and right positions in the image to be processed). Specifically, the electronic device can input the image to be processed into a pre-trained model and obtain the output of the model. The output of the model can be the vehicle region of each vehicle in the image.

[0052] In order to accurately determine the correlation between vehicles and parking spaces, the electronic device can identify the target vehicle among the vehicles in the image to be processed. The target vehicle can be any vehicle among the vehicles. The electronic device can also identify the vehicle annotation image corresponding to the target vehicle. The vehicle annotation image is an image in which the annotation information of the vehicle region of the target vehicle is added to the image to be processed.

[0053] S102: Perform key point detection on the vehicle-annotated image to determine the vehicle key points and parking space key points in the vehicle region of the target vehicle.

[0054] To accurately determine the correlation between vehicles and parking spaces, in this embodiment, the electronic device can perform key point detection on the vehicle-annotated image to identify each key point in the image. Each key point includes vehicle key points and parking space key points. The identified vehicle key points and parking space key points are the vehicle key points and parking space key points in the vehicle area of ​​the target vehicle. Vehicle key points include key points such as vehicle lights, rearview mirrors, and wheels, while parking space key points include key points at the vertices of the parking space lines.

[0055] It should be noted that some vehicle and parking space key points may be obscured by the surrounding environment. Therefore, the determined location information of vehicle and parking space key points is not the location information of all identifiable vehicle and parking space key points. The vehicle region of the target vehicle in the image to be processed must contain a vertex of the parking space line and the main body of the vehicle. Therefore, some vehicle and parking space key points in the vehicle region of the target vehicle can still be identified.

[0056] S103: Based on the vehicle key points and the parking space key points, determine the correlation between the target vehicle and the parking space; wherein, the parking space is the parking space associated with the parking space key points.

[0057] To accurately determine the correlation between vehicles and parking spaces, after identifying key points of both the vehicle and the parking space, the electronic device can determine the parking spaces associated with each key point. Specifically, the electronic device can locally store the corresponding associated parking spaces for each key point's location information. The electronic device can then determine the key parking space corresponding to the location information of that key point and determine the correlation between the target vehicle and the parking space. Specifically, it can determine if the target vehicle is parked in that parking space. Alternatively, it can determine whether the target vehicle is accurately parked in the parking space using methods such as projection described in existing technologies. Alternatively, it can use a pre-trained correlation determination model, inputting the location information of the vehicle and parking space key points into the model and obtaining its output, which is the probability value of the association between the target vehicle and the parking space.

[0058] After determining the correlation between vehicles and parking spaces, electronic devices or other equipment can determine whether a vehicle is parked in a parking space based on the correlation, thereby effectively managing the parking spaces.

[0059] In this embodiment, the electronic device detects the vehicle regions of each vehicle in the image to be processed and determines the vehicle annotation image corresponding to the target vehicle. The vehicle annotation image is an image in the image to be processed with annotation information of the vehicle region of the target vehicle added. The electronic device performs key point detection on the vehicle annotation image to determine the vehicle key points and parking space key points in the vehicle region of the target vehicle. Based on the determined vehicle key points and parking space key points, the correlation between the target vehicle and the parking space associated with the parking space key points is determined. Since a vehicle contains multiple vehicle key points and a parking space contains multiple parking space key points, even if some parts of the vehicle or parking space are occluded, the correlation between the vehicle and the parking space can still be accurately determined.

[0060] Example 2:

[0061] To accurately determine vehicle key points and parking space key points, based on the above embodiments, in this embodiment, the key point detection of the vehicle-annotated image to determine the vehicle key points and parking space key points in the vehicle region of the target vehicle includes:

[0062] The vehicle-labeled image is input into a pre-trained recognition model to obtain the vehicle key points and parking space key points in the vehicle region of the target vehicle output by the recognition model.

[0063] To accurately determine vehicle key points and parking space key points, the electronic device stores a pre-trained recognition model. The electronic device can input the vehicle annotation image corresponding to the target vehicle into the pre-trained recognition model. The annotation information added to the vehicle annotation image is the information of the vehicle region that identifies the target vehicle. The vehicle region corresponding to this annotation information is the region recognized by the recognition model. The electronic device obtains the output of the recognition model, which is the vehicle key points and parking space key points contained in the vehicle region of the target vehicle. Specifically, it can be the position information of the vehicle key points and parking space key points in the vehicle annotation image.

[0064] To accurately determine the correlation between a vehicle and a parking space, based on the above embodiments, in this embodiment, determining the correlation between the target vehicle and the parking space based on the vehicle's key points and the parking space's key points includes:

[0065] Based on the location information of the parking space key points in the image to be processed, determine the parking space associated with the parking space key points;

[0066] The target vehicle is determined to be a vehicle associated with the parking space.

[0067] In this embodiment of the application, the specific correlation between vehicles and parking spaces included in the vehicle area of ​​the target vehicle is determined by the electronic device after acquiring the vehicle key points and parking space key points. The electronic device can determine the parking space corresponding to the vehicle area of ​​the target vehicle based on the position information of the parking space key points in the vehicle annotation image of the vehicle area of ​​the target vehicle. Specifically, the electronic device stores the position information corresponding to the parking space key points of each parking space in the image acquired by the acquisition device that sends the image to be processed. Specifically, a preset number of parking space key points corresponding to each parking space can be stored. If the position information of the parking space key points acquired by the electronic device in the vehicle annotation image matches the position information corresponding to a certain parking space, it can be determined that the parking space is the parking space associated with the vehicle key points, that is, the parking space corresponding to the vehicle area of ​​the target vehicle.

[0068] For example, the location information of two parking space key points in the image is determined by the recognition model. For each parking space, the location information corresponding to four parking space key points is stored. If the two location information are consistent with two of the four location information of a certain parking space, it means that the location information of the obtained parking space key points in the vehicle annotation image matches the location information corresponding to that parking space. Then, it is determined that the parking space is the parking space associated with the vehicle key points.

[0069] After identifying the parking space associated with the vehicle's key points, the electronic equipment can determine the target vehicle, which is the vehicle parked in that space. This establishes the correlation between the vehicle and the parking space. The vehicle's key points are essentially some of the key points of the target vehicle.

[0070] Example 3:

[0071] To accurately determine vehicle key points and parking space key points, based on the above embodiments, in this embodiment, the step of inputting the vehicle-annotated image into a pre-trained recognition model to obtain the vehicle key points and parking space key points in the vehicle region of the target vehicle output by the recognition model includes:

[0072] The vehicle-labeled image is input into a pre-trained recognition model to obtain the vehicle key points and parking space key points in the vehicle region of the target vehicle output by the recognition model, as well as the offsets corresponding to the vehicle key points and parking space key points; wherein, the offsets are the offsets of the vehicle key points and parking space key points relative to the center point of their respective vehicle regions.

[0073] In this embodiment, the electronic device can input a vehicle-annotated image into a pre-trained recognition model and obtain the output of the recognition model. The output of the recognition model is the vehicle key points and parking space key points in the vehicle region of the target vehicle corresponding to the vehicle-annotated image, as well as the offsets corresponding to the vehicle key points and parking space key points. The offsets are the offsets of each vehicle key point and parking space key point relative to the center point of the vehicle region of its corresponding vehicle, and include both offset distance and offset angle. Specifically, if a key point is a vehicle key point, the offset output by the recognition model for that key point is the offset of that key point relative to the center point of the vehicle region of the vehicle to which the key point belongs. If a key point is a parking space key point, the offset output by the recognition model for that key point is the offset of that key point relative to the center point of the vehicle region of the vehicle parked in the parking space to which the key point belongs.

[0074] To accurately determine vehicle key points and parking space key points, based on the above embodiments, in this embodiment, after determining the vehicle key points and parking space key points in the vehicle area of ​​the target vehicle, and before determining the correlation between the target vehicle and the parking space based on the vehicle key points and parking space key points, the method further includes:

[0075] For each vehicle key point and parking space key point, based on the position information of the key point in the image to be processed and the offset of the key point relative to the center point of its corresponding vehicle area, the position information of the center point of the vehicle area corresponding to the key point in the image to be processed is determined. It is then determined whether the position information is located in the area where the center point of the target vehicle's vehicle area is located. If not, the key point is deleted.

[0076] In practical applications, the vehicle area of ​​a target vehicle may contain key points of other vehicles or key points of other parking spaces. Therefore, in this embodiment, the electronic device can delete key points of other vehicles and key points of other parking spaces from the vehicle annotation image.

[0077] Specifically, for each identified vehicle key point and parking space key point, the electronic device can determine the position information of the center point of the vehicle area corresponding to the key point, based on the key point's position information in the image and the offset of the key point relative to the center point of the vehicle area of ​​the corresponding vehicle. Specifically, given the position information of a certain point and the offset of another point relative to that point, how to determine the position information of the other point is existing technology and will not be elaborated here. If the key point is a vehicle key point, then the vehicle area of ​​the vehicle corresponding to the key point is the vehicle area corresponding to the vehicle to which the key point belongs. If the key point is a parking space key point, then the vehicle area of ​​the vehicle corresponding to the key point is the vehicle area corresponding to the vehicle parked in the parking space to which the key point belongs.

[0078] For each key point, after determining the location information of the center point of the vehicle area corresponding to the key point, the electronic device can determine whether the vehicle area corresponding to the key point is the vehicle area of ​​the target vehicle based on the determined location information and the location information of the center point of the target vehicle's vehicle area. Specifically, the electronic device can determine an area with a preset shape centered on the center point of the target vehicle's vehicle area. This area is the area where the center point of the vehicle area is located. For example, the electronic device can determine a circular area with the center point of the target vehicle's vehicle area as the center and a preset length as the radius. The electronic device can determine whether the location information of the center point of the vehicle area corresponding to the key point is located within the area where the center point of the target vehicle's vehicle area is located. If it is not located within the area where the center point of the target vehicle's vehicle area is located, it means that the key point is a vehicle key point of another vehicle or a parking space key point of another parking space, and then the key point is deleted.

[0079] In this embodiment, the electronic device can determine whether the vehicle key points and parking space key points identified by the recognition model are key points corresponding to the target vehicle based on the geometric information of the offset corresponding to the vehicle key points and parking space key points, as well as the position information of the vehicle key points and parking space key points in the vehicle annotation image. It can also block other key points that are not the target vehicle or the parking space corresponding to the target vehicle, so that the key points of other vehicles or other parking spaces will not be identified as vehicle key points and parking space key points in the vehicle area of ​​the target vehicle due to the same features. That is, it will not identify the components of different vehicles as the components of the same vehicle due to the same features. This solves the problem of vehicle components not being able to be bound to the vehicle from the root. It is equivalent to recognizing and binding the parking space key points and vehicle key points together to achieve end-to-end positioning of the vehicle and parking space. Furthermore, the geometric information is used to further improve the accuracy of vehicle key point and parking space key point recognition in the vehicle area.

[0080] Figure 2This is a schematic diagram of an image to be processed, provided as an embodiment of this application.

[0081] Depend on Figure 2 It can be seen that three vehicle regions containing vehicles can be identified in the image to be processed, and Figure 2 The three circular areas in the diagram represent the areas where the center points of the three vehicle zones are located, with the middle ( Figure 2 The vehicle area shown in the middle contains the rearview mirrors of two other vehicles. If only the key points of the vehicles and parking spaces within this middle vehicle area are determined, the rearview mirrors of the other two vehicles will definitely be detected, leading to inaccurate determination of the correlation between vehicles and parking spaces. Although the rearview mirrors of the other two vehicles are within the middle vehicle area, by combining the offset corresponding to the vehicle key points, it can be determined that the center point of the vehicle area corresponding to the rearview mirrors of the other two vehicles is not located within the area where the center point of the middle vehicle area is located. Therefore, the rearview mirrors of the other two vehicles can be removed.

[0082] Among them, the offsets corresponding to the vehicle key points and parking space key points output by the recognition model can be the polar coordinates of the center point of the vehicle area corresponding to the key point in a coordinate system established with the key point as the pole of the polar coordinates and the preset direction as the polar axis.

[0083] In this embodiment, before training the recognition model, the electronic device can, for each vehicle, determine the offset of each vehicle key point and each parking space key point in the vehicle's parking space relative to the center point of the vehicle's vehicle region in the sample image. The electronic device can train the original recognition model based on the offsets of the vehicle key points and parking space key points, as well as the position information of the vehicle key points and parking space key points in the sample image. Specifically, when determining each offset, the first position information of each vehicle key point and each parking space key point in the sample image, and the second position information of the vehicle's center point in the image can be determined, and the corresponding offset is determined based on the first position information and the second position information.

[0084] Figure 3 This is a schematic diagram of the center point of the vehicle area corresponding to each determined key point, provided as an embodiment of this application.

[0085] Depend on Figure 3 It can be seen that, Figure 3 The key points of the vehicles and parking spaces marked in the image correspond to the center point of their respective vehicle areas, all located at... Figure 3 Within the circular area shown, Figure 3 The central circular area is the region where the center point of the target vehicle's vehicle area is located. Figure 3The vehicle key points and parking space key points marked in the text are not the vehicle key points of other vehicles, nor are they the parking space key points of other parking spaces.

[0086] In this embodiment of the application, the electronic device deletes the vehicle key points of other vehicles and the parking space key points of other parking spaces, thereby accurately determining the correlation between vehicles and parking spaces.

[0087] Example 4:

[0088] To accurately determine the location information of parking space key points in the image, based on the above embodiments, in this embodiment, after detecting the vehicle regions of each vehicle in the image to be processed and before determining the vehicle annotation image corresponding to the target vehicle, the method further includes:

[0089] The vehicle area of ​​each detected vehicle is magnified by a preset factor;

[0090] For the magnified vehicle area, perform the subsequent step of determining the vehicle annotation image corresponding to the target vehicle.

[0091] In practical applications, when a vehicle is parked in a parking space, the area corresponding to the parking space in the image to be processed is usually larger than the area corresponding to the vehicle. Therefore, the identified vehicle area of ​​the target vehicle may lack some parking space key points. In order to ensure that the identified vehicle area of ​​the target vehicle contains parking space key points, in this embodiment, the electronic device can magnify the vehicle area of ​​the target vehicle by a preset factor, wherein the preset factor is a factor greater than 1, for example, it can be 1.5 times. Usually, the preset factor is not too large. The electronic device performs the subsequent step of determining the vehicle annotation image corresponding to the target vehicle for the magnified vehicle area.

[0092] Figure 4 This is a schematic diagram of key points of a vehicle and key points of a parking space provided in an embodiment of this application.

[0093] Figure 4 The vehicle area marked with the target vehicle. Figure 4 The points marked in the text are the vehicle key points corresponding to the vehicle area, and the parking space key points of the parking spaces where the corresponding vehicles are parked. Figure 4 It can be seen that the vehicle area of ​​the target vehicle does not include the key point of the parking space where the vehicle is parked.

[0094] In this embodiment, the electronic device magnifies the vehicle area of ​​the target vehicle by a preset factor, thereby including the key points of the parking space where the vehicle is parked in the magnified vehicle area, and thus accurately determines the key points of the parking space corresponding to the target vehicle.

[0095] To accurately obtain the vehicle regions of each vehicle in the image to be processed, based on the above embodiments, in this embodiment, the detection of the vehicle regions of each vehicle in the image to be processed includes:

[0096] The image to be processed is input into a pre-trained vehicle recognition model to obtain the vehicle regions of each vehicle in the image to be processed, as output by the vehicle recognition model.

[0097] In this embodiment of the application, the electronic device has a pre-trained vehicle recognition model stored in it. After the electronic device acquires the image to be processed acquired by the acquisition device, it can input the acquired image to be processed into the pre-trained vehicle recognition model and obtain the output of the vehicle recognition model. The output image of the vehicle recognition model includes the vehicle regions of each vehicle.

[0098] To accurately determine the correlation between vehicles and parking spaces, based on the above embodiments, in this embodiment, the step of inputting the vehicle-annotated image into a pre-trained recognition model to obtain the vehicle key points and parking space key points in the vehicle region of the target vehicle output by the recognition model, as well as the offsets corresponding to the vehicle key points and parking space key points, includes:

[0099] The vehicle-labeled image is input into the first and second convolutional layers of a pre-trained recognition model. The first convolutional layer processes the vehicle-labeled image to obtain the types of vehicle key points and parking space key points contained in the vehicle region of the target vehicle in the vehicle-labeled image, as well as their position information in the vehicle-labeled image. The second convolutional layer processes the vehicle-labeled image to obtain the types of vehicle key points and parking space key points contained in the vehicle region of the target vehicle in the vehicle-labeled image, as well as their corresponding offsets.

[0100] The network layer of the recognition model determines, based on the processing results of the first convolutional layer and the second convolutional layer, the vehicle key points and parking space key points contained in the vehicle region of the target vehicle in the vehicle annotation image, as well as the offsets corresponding to the vehicle key points and parking space key points.

[0101] In this embodiment, the electronic device inputs a vehicle-annotated image into a pre-trained recognition model. The first and second convolutional layers of the recognition model process the vehicle-annotated image. Specifically, the first convolutional layer processes the vehicle-annotated image to obtain the types of vehicle key points and parking space key points in the vehicle region of the target vehicle in the vehicle-annotated image, as well as their position information in the vehicle-annotated image. The types of vehicle key points and parking space key points include wheels, headlights, and the upper left corner of the parking space (here, up, down, left, and right refer to the up, down, left, and right in the actual application scenario). The second convolutional layer processes the vehicle-annotated image to obtain the types of vehicle key points and parking space key points contained in the vehicle region of the target vehicle in the vehicle-annotated image and their corresponding offsets. The offset refers to the offset of the vehicle key point and parking space key point relative to the center point of the vehicle region of their respective vehicle.

[0102] The first and second convolutional layers of the recognition model input their respective processing results into the network layer of the recognition model. Specifically, the first convolutional layer inputs the types of vehicle key points and parking space key points in the vehicle region of the target vehicle and their position information in the vehicle annotation image into the network layer of the recognition model. The second convolutional layer inputs the types of vehicle key points and parking space key points in the vehicle region of the target vehicle and their corresponding offsets into the network layer of the recognition model. The network layer of the recognition model processes the received processing results from the first and second convolutional layers. The electronic device acquires the output of the network layer, which is the output of the recognition model. This output includes the vehicle key points and parking space key points contained in the vehicle region of the target vehicle in the vehicle annotation image, as well as the corresponding offsets of the vehicle key points and parking space key points.

[0103] In this embodiment of the application, in order to accurately determine the vehicle key points and parking space key points contained in the vehicle region of the target vehicle in the vehicle annotation image, as well as the offsets corresponding to the vehicle key points and parking space key points, the recognition model can first perform feature extraction and other operations on the vehicle annotation image through other layers after receiving the vehicle annotation image, so as to facilitate the processing of the vehicle annotation image by the first convolutional layer and the second convolutional layer.

[0104] Figure 5 This is a schematic diagram illustrating the processing procedure of a recognition model provided in an embodiment of this application.

[0105] Depend on Figure 5As can be seen, the electronic device inputs the vehicle-annotated image into the recognition model. The backbone, self-attention layer (SA), and feature pyramid network (FPN) of the recognition model process the vehicle-annotated image. Specifically, how the backbone, SA, and FPN process the image is existing technology and will not be elaborated here. After the FPN processes the image, the processing results are input into the first convolutional layer and the second convolutional layer, respectively. The first convolutional layer obtains the type and position information of the vehicle key points and parking space key points in the vehicle-annotated image. The second convolutional layer obtains the type and corresponding offset of the vehicle key points and parking space key points. The first and second convolutional layers input the processing results into the network layer, and the network layer obtains the position information and corresponding offset of the vehicle key points and parking space key points in the image.

[0106] Example 5:

[0107] Figure 6 This application provides a schematic diagram of a vehicle parking space correlation determination device, which includes:

[0108] The detection and determination module 601 is used to detect the vehicle regions of each vehicle in the image to be processed; determine the vehicle annotation image corresponding to the target vehicle; wherein, the vehicle annotation image is an image in the image to be processed with annotation information of the vehicle region of the target vehicle added, and the target vehicle is the vehicle in the image to be processed; perform key point detection on the vehicle annotation image to determine the vehicle key points and parking space key points in the vehicle region of the target vehicle.

[0109] The processing module 602 is used to determine the correlation between the target vehicle and the parking space based on the vehicle key points and the parking space key points; wherein the parking space is the parking space associated with the parking space key points.

[0110] Furthermore, the detection and determination module 601 is specifically used to input the vehicle annotation image into a pre-trained recognition model to obtain the vehicle key points and parking space key points in the vehicle region of the target vehicle output by the recognition model.

[0111] Furthermore, the processing module 602 is specifically used to determine the parking space associated with the parking space key point based on the position information of the parking space key point in the image to be processed; and to determine that the target vehicle is a vehicle associated with the parking space.

[0112] Furthermore, the detection and determination module 601 is specifically used to input the vehicle annotation image into a pre-trained recognition model, and obtain the vehicle key points and parking space key points in the vehicle region of the target vehicle output by the recognition model, as well as the offsets corresponding to the vehicle key points and parking space key points; wherein, the offsets are the offsets of the vehicle key points and parking space key points relative to the center point of their respective vehicle regions.

[0113] Furthermore, the processing module 602 is also used to, for each vehicle key point and parking space key point, determine the position information of the center point of the vehicle area corresponding to the key point in the image to be processed based on the position information of the key point in the image to be processed and the offset of the key point relative to the center point of its corresponding vehicle area, determine whether the position information is located in the area where the center point of the vehicle area of ​​the target vehicle is located, and if not, delete the key point.

[0114] Furthermore, the detection and determination module 601 is also used to magnify the vehicle area of ​​each detected vehicle by a preset factor; and for the magnified vehicle area, to perform the subsequent step of determining the vehicle annotation image corresponding to the target vehicle.

[0115] Furthermore, the detection and determination module 601 is specifically used to input the image to be processed into a pre-trained vehicle recognition model to obtain the vehicle regions of each vehicle in the image to be processed output by the vehicle recognition model.

[0116] Example 5:

[0117] Based on the above embodiments, this application also provides an electronic device. Figure 7 This application provides a schematic diagram of an electronic device structure, such as... Figure 7 As shown, it includes: processor 701, communication interface 702, memory 703 and communication bus 704, wherein processor 701, communication interface 702 and memory 703 communicate with each other through communication bus 704.

[0118] The memory 703 stores a computer program. When the program is executed by the processor 701, the processor 701 performs the following steps:

[0119] Detect vehicle regions of each vehicle in the image to be processed; determine the vehicle annotation image corresponding to the target vehicle; wherein, the vehicle annotation image is an image in the image to be processed with annotation information of the vehicle region of the target vehicle added, and the target vehicle is the vehicle in the image to be processed;

[0120] Key point detection is performed on the vehicle-annotated image to determine the vehicle key points and parking space key points in the vehicle region of the target vehicle.

[0121] Based on the vehicle key points and the parking space key points, the correlation between the target vehicle and the parking space is determined; wherein, the parking space is the parking space associated with the parking space key points.

[0122] Furthermore, the processor 701 is specifically used to input the vehicle annotation image into a pre-trained recognition model to obtain vehicle key points and parking space key points in the vehicle region of the target vehicle output by the recognition model.

[0123] Furthermore, the processor 701 is specifically used to determine the parking space associated with the parking space key point based on the position information of the parking space key point in the image to be processed;

[0124] The target vehicle is determined to be a vehicle associated with the parking space.

[0125] Furthermore, the processor 701 is specifically used to input the vehicle annotation image into a pre-trained recognition model to obtain the vehicle key points and parking space key points in the vehicle region of the target vehicle output by the recognition model, as well as the offsets corresponding to the vehicle key points and parking space key points; wherein, the offsets are the offsets of the vehicle key points and parking space key points relative to the center point of their respective vehicle regions.

[0126] Furthermore, the processor 701 is also configured to, for each vehicle key point and parking space key point, determine the position information of the center point of the vehicle area corresponding to the key point in the image to be processed based on the position information of the key point in the image to be processed and the offset of the key point relative to the center point of its corresponding vehicle area, determine whether the position information is located in the area where the center point of the vehicle area of ​​the target vehicle is located, and if not, delete the key point.

[0127] Furthermore, the processor 701 is also used to magnify the vehicle area of ​​each detected vehicle by a preset factor;

[0128] For the magnified vehicle area, perform the subsequent step of determining the vehicle annotation image corresponding to the target vehicle.

[0129] Furthermore, the processor 701 is specifically used to input the image to be processed into a pre-trained vehicle recognition model to obtain the vehicle regions of each vehicle in the image to be processed output by the vehicle recognition model.

[0130] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0131] The communication interface 702 is used for communication between the aforementioned electronic device and other devices. The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0132] The processors mentioned above can be general-purpose processors, including central processing units, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0133] Example 6:

[0134] Based on the above embodiments, this application also provides a computer-readable storage medium storing a computer program executable by a processor. When the program runs on the processor, it causes the processor to perform the following steps:

[0135] Detect vehicle regions of each vehicle in the image to be processed; determine the vehicle annotation image corresponding to the target vehicle; wherein, the vehicle annotation image is an image in the image to be processed with annotation information of the vehicle region of the target vehicle added, and the target vehicle is the vehicle in the image to be processed;

[0136] Key point detection is performed on the vehicle-annotated image to determine the vehicle key points and parking space key points in the vehicle region of the target vehicle.

[0137] Based on the vehicle key points and the parking space key points, the correlation between the target vehicle and the parking space is determined; wherein, the parking space is the parking space associated with the parking space key points.

[0138] In one possible implementation, the step of performing key point detection on the vehicle-annotated image to determine vehicle key points and parking space key points in the vehicle region of the target vehicle includes:

[0139] The vehicle-labeled image is input into a pre-trained recognition model to obtain the vehicle key points and parking space key points in the vehicle region of the target vehicle output by the recognition model.

[0140] In one possible implementation, determining the correlation between the target vehicle and the parking space based on the vehicle key points and the parking space key points includes:

[0141] Based on the location information of the parking space key points in the image to be processed, determine the parking space associated with the parking space key points;

[0142] The target vehicle is determined to be a vehicle associated with the parking space.

[0143] In one possible implementation, the step of inputting the vehicle-annotated image into a pre-trained recognition model to obtain vehicle key points and parking space key points in the vehicle region of the target vehicle output by the recognition model includes:

[0144] The vehicle-labeled image is input into a pre-trained recognition model to obtain the vehicle key points and parking space key points in the vehicle region of the target vehicle output by the recognition model, as well as the offsets corresponding to the vehicle key points and parking space key points; wherein, the offsets are the offsets of the vehicle key points and parking space key points relative to the center point of their respective vehicle regions.

[0145] In one possible implementation, after determining the vehicle key points and parking space key points in the vehicle area of ​​the target vehicle, and before determining the correlation between the target vehicle and the parking space based on the vehicle key points and the parking space key points, the method further includes:

[0146] For each vehicle key point and parking space key point, based on the position information of the key point in the image to be processed and the offset of the key point relative to the center point of its corresponding vehicle area, the position information of the center point of the vehicle area corresponding to the key point in the image to be processed is determined. It is then determined whether the position information is located in the area where the center point of the target vehicle's vehicle area is located. If not, the key point is deleted.

[0147] In one possible implementation, after detecting the vehicle regions of each vehicle in the image to be processed and before determining the vehicle annotation image corresponding to the target vehicle, the method further includes:

[0148] The vehicle area of ​​each detected vehicle is magnified by a preset factor;

[0149] For the magnified vehicle area, perform the subsequent step of determining the vehicle annotation image corresponding to the target vehicle.

[0150] In one possible implementation, the detection of vehicle regions for each vehicle in the image to be processed includes:

[0151] The image to be processed is input into a pre-trained vehicle recognition model to obtain the vehicle regions of each vehicle in the image to be processed, as output by the vehicle recognition model.

[0152] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0153] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0154] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0155] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0156] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for determining the correlation of vehicle parking spaces, characterized in that, The method includes: Detect vehicle regions of each vehicle in the image to be processed; determine the vehicle annotation image corresponding to the target vehicle; wherein, the vehicle annotation image is an image in the image to be processed with annotation information of the vehicle region of the target vehicle added, and the target vehicle is the vehicle in the image to be processed; The vehicle-labeled image is input into a pre-trained recognition model to obtain the vehicle key points and parking space key points in the vehicle region of the target vehicle output by the recognition model, as well as the offsets corresponding to the vehicle key points and parking space key points; wherein, the offsets are the offsets of the vehicle key points and parking space key points relative to the center point of their respective vehicle regions; the offsets include offset distance and offset angle. For each vehicle key point and parking space key point, based on the position information of the key point in the image to be processed and the offset of the key point relative to the center point of its corresponding vehicle area, the position information of the center point of the vehicle area corresponding to the key point in the image to be processed is determined. It is then determined whether the position information is located in the area where the center point of the vehicle area of ​​the target vehicle is located. If not, the key point is deleted to block other key points that are not the target vehicle or not corresponding to the target vehicle. Based on the retained vehicle key points and parking space key points, the correlation between the target vehicle and the parking space is determined; wherein, the parking space is the parking space associated with the parking space key point.

2. The method according to claim 1, characterized in that, The determination of the correlation between the target vehicle and the parking space based on the vehicle key points and the parking space key points includes: Based on the location information of the parking space key points in the image to be processed, determine the parking space associated with the parking space key points; The target vehicle is determined to be a vehicle associated with the parking space.

3. The method according to claim 1, characterized in that, After detecting the vehicle regions of each vehicle in the image to be processed, and before determining the vehicle annotation image corresponding to the target vehicle, the method further includes: The vehicle area of ​​each detected vehicle is magnified by a preset factor; For the magnified vehicle area, perform the subsequent step of determining the vehicle annotation image corresponding to the target vehicle.

4. The method according to claim 1, characterized in that, The vehicle regions of each vehicle in the image to be processed include: The image to be processed is input into a pre-trained vehicle recognition model to obtain the vehicle regions of each vehicle in the image to be processed, as output by the vehicle recognition model.

5. A device for determining the correlation of vehicle parking spaces, characterized in that, The device includes: The processing module is used to detect the vehicle regions of each vehicle in the image to be processed; and determine the vehicle annotation image corresponding to the target vehicle; wherein, the vehicle annotation image is an image in the image to be processed with annotation information of the vehicle region of the target vehicle added, and the target vehicle is the vehicle in the image to be processed; The detection and determination module is used to input the vehicle-annotated image into a pre-trained recognition model, obtain vehicle key points and parking space key points in the vehicle region of the target vehicle output by the recognition model, and the corresponding offsets of the vehicle key points and parking space key points; wherein, the offset is the offset of the vehicle key point and parking space key point relative to the center point of their respective vehicle region; the offset includes offset distance and offset angle; for each vehicle key point and parking space key point, based on the position information of the key point in the image to be processed and the offset of the key point relative to the center point of its corresponding vehicle region, the position information of the center point of the vehicle region corresponding to the key point in the image to be processed is determined, and it is determined whether the position information is located in the area where the center point of the target vehicle's vehicle region is located. If not, the key point is deleted to block other key points that are not the target vehicle or not corresponding to the target vehicle; based on the retained vehicle key points and parking space key points, the correlation between the target vehicle and the parking space is determined; wherein, the parking space is the parking space associated with the parking space key point.

6. An electronic device, characterized in that, The electronic device includes a processor that executes a computer program stored in a memory to implement the steps of the vehicle parking space correlation determination method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the vehicle parking space correlation determination method as described in any one of claims 1-4.

Images