Parking space detection method and device, sample generation method and device, vehicle and storage medium

Abstract

The invention provides a parking space detection method and device, a sample generation method and device, a vehicle and a storage medium, and the method comprises the steps: carrying out the screening of long tail data of a complex scene through a visual detection model disposed in a server, so as to determine a training sample for training an initial parking space detection model, the diversity and accuracy of the training samples are improved, the recognition accuracy of the parking space detection model obtained through training is further improved, parking space recognition is carried out based on the parking space detection model, and the recognition accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a parking space detection method, a sample generation method, a device, a vehicle and a storage medium. Background Art

[0002] With the popularization of autonomous driving and assisted driving, more and more vehicles are equipped with automatic parking technology, which helps drivers find suitable parking locations and complete parking.

[0003] However, as vehicle driving scenarios become more and more complex, the accuracy of parking space recognition becomes poor, resulting in the inability to park in the identified parking spaces. Therefore, how to improve the accuracy of parking space detection is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0005] To this end, the present application proposes a parking space detection method, a sample generation method, a device, a vehicle and a storage medium to improve the accuracy of parking space detection.

[0006] Another aspect of the present application provides a parking space detection method, including:

[0007] Acquire a first parking area image to be detected;

[0008] A parking space detection model is used to identify the parking spaces in the first parking area image to determine whether there is a first parking space that can be parked; wherein, the training samples used for training the parking space detection model are obtained by using a visual detection model in a server to identify a second parking area image collected by the vehicle, and in response to identifying the presence of parking obstacle information around the second parking space in the second parking area image, the second parking space is a parking space that can be parked that is obtained by the vehicle identifying the second parking area image based on the initial parking space detection model.

[0009] In one aspect, an embodiment of the present application provides a sample generation method, including:

[0010] Acquire a first parking area image collected by the vehicle; wherein the first parking area image includes a first parking space that the vehicle can park in and that is identified based on an initial parking space detection model;

[0011] Using a visual detection model to identify the first parking area image;

[0012] In response to identifying the presence of parking obstacle information around the first parking space, a training sample of the initial parking space detection model is generated according to the first parking area image.

[0013] Another aspect of the present application provides a parking space detection device, including:

[0014] An acquisition module, used for acquiring an image of a first parking area to be detected;

[0015] a determination module, configured to identify the parking spaces in the first parking area image by using a parking space detection model, and determine whether there is a first parking space available for parking; wherein the training samples used for training the parking space detection model are obtained by identifying a second parking area image collected by the vehicle by using a visual detection model in a server, and in response to identifying the presence of parking obstacle information around the second parking space in the second parking area image, the determination module is generated according to the second parking area image, wherein the second parking space is a parking space available for parking obtained by the vehicle through the recognition of the second parking area image based on the initial parking space detection model.

[0016] Another aspect of the present application provides a sample generation device, including:

[0017] An acquisition module, configured to acquire a first parking area image collected by the vehicle; wherein the first parking area image includes a first parking space that the vehicle can park in, which is identified based on an initial parking space detection model;

[0018] A recognition module, configured to recognize the first parking area image using a visual detection model;

[0019] A generating module is used for generating a training sample of the initial parking space detection model according to the first parking area image in response to identifying the parking obstacle information around the first parking space.

[0020] Another aspect of the present application provides a vehicle, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the above aspect is implemented.

[0021] Another aspect of the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method described in the aforementioned aspect is implemented.

[0022] Another aspect of the present application provides a computer program product on which a computer program is stored. When the program is executed by a processor, the method described in the above aspect is implemented.

[0023] The parking space detection method, sample generation method, device, vehicle and storage medium proposed in the present application screen the long-tail data of complex scenes through the visual detection model set in the server to determine the training samples for training the initial parking space detection model, thereby improving the diversity and accuracy of the training samples, and further improving the recognition accuracy of the trained parking space detection model, so as to perform parking space recognition based on the parking space detection model and improve the recognition accuracy.

[0024] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0026] Figure 1 A schematic diagram of a parking space detection method provided in an embodiment of the present application;

[0027] Figure 2 A schematic diagram of a parking scenario provided in an embodiment of the present application;

[0028] Figure 3 A schematic diagram of a flow chart of a sample generation method provided in an embodiment of the present application;

[0029] Figure 4 A schematic diagram of a flow chart of another sample generation method provided in an embodiment of the present application;

[0030] Figure 5 A schematic diagram of a parking area image provided in an embodiment of the present application;

[0031] Figure 6 A schematic diagram of the structure of a parking space detection device provided in an embodiment of the present application;

[0032] Figure 7 A schematic diagram of the structure of a sample generation device provided in an embodiment of the present application;

[0033] Figure 8 It is a structural schematic diagram of a vehicle proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0034] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0035] The following describes the parking space detection method, sample generation method, device, vehicle and storage medium of the embodiments of the present application with reference to the accompanying drawings.

[0036] Figure 1 A flowchart of a parking space detection method provided in an embodiment of the present application.

[0037] The embodiment of the present application takes the parking space detection method being configured in a parking space detection device as an example. The parking space detection device can be applied to any vehicle-mounted device so that the vehicle-mounted device can perform a parking space detection function.

[0038] like Figure 1 As shown, the method may include the following steps:

[0039] Step 101: Acquire a first parking area image to be detected.

[0040] The parking area refers to a designated area that provides legal parking space for motor vehicles, including at least one parking space. The first parking area image refers to an image acquired from the parking area at a collection time.

[0041] Step 102: Use a parking space detection model to identify parking spaces in the first parking area image to determine whether there is a first parking space available for parking.

[0042] The parking space detection model is set at the vehicle end, and the training samples used for training the parking space detection model are generated based on the second parking area image in response to the identification of parking obstacle information around the second parking space in the second parking area image, wherein the second parking space is a parking space that can be parked obtained by the vehicle based on the initial parking space detection model for the second parking area image. The long-tail data of complex scenes is screened through the visual detection model set in the server, which improves the diversity and accuracy of the training samples, thereby improving the recognition accuracy of the parking space detection model, thereby performing parking space recognition based on the parking space detection model and improving the recognition accuracy.

[0043] Among them, the training method for generating training samples of the parking space detection model will be described in detail in subsequent embodiments and will not be repeated in this experiment.

[0044] In the parking space detection method of the embodiment of the present application, the long-tail data of complex scenes is screened through the visual detection model set in the server to determine the training samples for training the initial parking space detection model, thereby improving the diversity and accuracy of the training samples, and further improving the recognition accuracy of the trained parking space detection model, so that parking space recognition is performed based on the parking space detection model, and the recognition accuracy is improved.

[0045] Based on the above embodiment, the embodiment of the present application provides another parking space detection method, wherein there are multiple first parking spaces, and after using the parking space detection model to identify the parking spaces in the first parking area image and determining whether there is a first parking space that can be parked, the method further includes: in response to the existence of multiple first parking spaces, sending the first parking area image collected by the vehicle to the server. As an example, Figure 2 A schematic diagram of a parking scenario provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the parking area includes 5 parking spaces, 3 of which are already occupied by vehicles, and there are 2 first parking spaces available for parking. The first parking area image is used by the server to recognize the first parking area image using a visual detection model to obtain parking decision information of each first parking space, and the parking decision information includes whether there are obstacles around the first parking space, that is, whether the first parking space can be parked.

[0046] Then, the parking decision information of each first parking space sent by the server is obtained, and a first candidate parking space with no obstacles around it is determined according to the parking decision information of each first parking space. Since there are no obstacles around the first candidate parking space, the first candidate parking space is a parking space that can be parked in. Then, according to the first candidate parking space, a first target parking space to be parked in is determined, wherein, as an implementation manner, if there is one first candidate parking space, the first candidate parking space is used as the first target parking space to be parked in; as another implementation manner, if there are multiple first candidate parking spaces, according to the sizes of the multiple first candidate parking spaces, a second candidate parking space whose parking space size matches the size of the vehicle is determined, wherein matching means that the parking space size is greater than or equal to the size of the vehicle, the second candidate parking space is used as the first target parking space, and the vehicle is controlled to park in the first target parking space, so as to match the appropriate parking space based on the vehicle size, improve the efficiency of vehicle parking, and reduce the difficulty of vehicle parking.

[0047] In the parking space detection method of the embodiment of the present application, the parking space detection model is updated and trained through training samples, so that the updated and trained parking space detection model has the ability to recognize special obstacle scenes, and can accurately identify whether there is a parking space that can be parked, thereby improving the accuracy of parking space recognition, and further improving the safety of parking.

[0048] Based on the above embodiments, Figure 3 A schematic diagram of a flow chart of a sample generation method provided in an embodiment of the present application.

[0049] The embodiment of the present application takes the sample generation method being configured in a sample generation device as an example. The sample generation device can be applied to a server, which is a locally set server or a cloud device server. The server can be a single server or a server cluster so that the server can perform a sample generation function.

[0050] like Figure 3 As shown, the method may include the following steps:

[0051] Step 301: Acquire a first parking area image collected by a vehicle.

[0052] The first parking area image includes a first parking space that can be parked and is identified by the initial parking space detection model.

[0053] In the embodiment of the present application, the first parking area image is an image of the parking area collected by the current vehicle, and "first" is used to identify the parking area image. As an implementation method, the first parking area image is collected by a camera on the vehicle side, for example, by a surround fisheye camera set on the vehicle side. The fisheye camera has a larger field of view and can collect images of a larger area.

[0054] Among them, the parking space detection model is a deep learning model arranged on the vehicle side, which is used to sense whether there are vacant parking spaces that can be parked in the parking scene. Among them, the initial parking space detection model can be an untrained parking space detection model, or a parking space detection model trained with samples of non-long-tail data scenes, that is, the training samples of the long-tail data scenes generated by the present application can be used for the first training of the initial parking space detection model, and can also be used for updating the training of the initial parking space detection model to improve the accuracy of detection. As an implementation method, the image of the first parking area of ​​the parking scene is collected by the image sensor on the vehicle side, and the image of the first parking area is recognized based on the initial parking space detection model set on the vehicle side to obtain the first parking space that can be parked. The first parking space that can be parked is determined by vehicle recognition, and the vehicle side believes that the first parking space is parkable. However, due to the limited computing power of the vehicle-side platform, it is impossible to deploy a model with a complex structure that requires a large computing power support. Therefore, for complex special scenes, the accuracy of parking space recognition is limited to a certain extent, that is, the detected first parking space that can be parked is not necessarily actually parkable.

[0055] Step 302: Use a visual detection model to identify the first parking area image.

[0056] As an example, the visual detection model can be a visual large language model with strong recognition and reasoning capabilities.

[0057] In the embodiment of the present application, the server is deployed in the cloud as an example. The server in the cloud has strong computing power and can deploy a visual detection model trained based on data of complex parking scenes, so as to identify the first parking area image through the visual detection model. As an implementation method, the recognition result can be determined based on the environmental information around each parking space in the first parking area image, the spatial information and the route information of parking in each parking space. The recognition result includes whether there is parking obstacle information around the first parking space identified by the vehicle end. The parking obstacle information refers to the obstacle information that affects the parking of the vehicle, that is, the vehicle cannot park in the first parking space due to the parking obstacle. Among them, the parking obstacle information around the first parking space includes parking obstacle information and path parking obstacle information. Parking obstacle information refers to the information of obstacles on the parking space of the first parking space, and path parking obstacle information refers to the information of obstacles on the parking path of the vehicle from the current position to the first parking space. The obstacle information includes the obstacle type, and can also include information such as obstacle size and position.

[0058] Step 303 : In response to identifying the presence of parking obstacle information around the first parking space, a training sample of an initial parking space detection model is generated according to the first parking area image.

[0059] In an embodiment of the present application, if the visual detection model in the cloud recognizes the parking obstacle information around the first parking space, it means that the initial parking space detection model did not recognize the parking obstacle information around the first parking space when identifying the first parking space. Therefore, the first parking space identified by the vehicle side is actually not parkable. Therefore, the first parking area image belongs to the long-tail data of the autonomous driving scene, which is difficult to obtain. The visual detection model set in the server is used for recognition, thereby realizing the business screening of the autonomous driving parking data. Furthermore, according to the first parking area image, a training sample of the initial parking space detection model is generated, and the training sample is used to train the initial parking space detection model to improve the accuracy of the parking space detection model in scene recognition, thereby improving the accuracy of parking space detection.

[0060] In the sample generation method of the embodiment of the present application, the visual detection model set in the server is used to identify the first parking area image. If parking obstacle information around the first parking space is identified, it is considered that the first parking space that the vehicle can park in according to the parking space detection model on the first parking area image is incorrect, and the first parking space is actually a parking space that cannot be parked in. Based on the first parking area image, a training sample of the parking space detection model is generated, and the complex scene in which the vehicle detection is inaccurate is determined by the visual detection model set in the server. The image of the scene is used for training the initial parking space detection model, which improves the diversity and accuracy of the training samples, thereby improving the recognition accuracy of the trained parking space detection model and improving parking safety.

[0061] Based on the above embodiments, Figure 4 A flow chart of another sample generation method provided in an embodiment of the present application is shown as follows: Figure 4 As shown, the method comprises the following steps:

[0062] Step 401: Acquire a first parking area image collected by a vehicle.

[0063] Among them, step 401 can refer to the relevant explanations in the above-mentioned embodiments, and the principle is the same, which will not be repeated here.

[0064] Step 402, obtaining a setting prompt word for the parking scene.

[0065] Among them, the set prompt word is a set prompt word used to assist the visual large language model in identifying obstacle information of obstacles around the first parking space that affect vehicle parking when recognizing the first parking area image. For example, the prompt word is: whether there are obstacles around the first parking space that affect parking.

[0066] Step 403 : According to the target prompt word, the visual large language model is used to recognize the first parking area image to obtain parking obstacle information around the first parking space.

[0067] Among them, the parking obstacle information around the first parking space includes the parking obstacle information on the first parking space and the parking obstacle information on the parking path of the target vehicle from the current position into the first parking space, thereby realizing the determination of various obstacle information around the vehicle that affects parking and improving the accuracy of the obstacle information determination.

[0068] The visual detection model is obtained by pre-training. As an implementation method, a parking area sample image is obtained, wherein the parking obstacle information around the parking space is annotated in the parking area sample image, and the parking area sample image is input into the initial visual detection model to obtain the predicted parking obstacle information of the parking space. According to the difference between the predicted parking obstacle information and the annotated parking obstacle information, a loss function is determined, and the initial visual detection model is trained according to the loss function to obtain a trained visual detection model. The training process of the visual detection model can be repeated multiple times, and the parking area sample images of different scenes are used for training. When the loss function is less than a set threshold or the training is set times, the visual detection model training is completed, and the trained visual detection model is obtained. The trained visual detection model has a strong reasoning ability and can identify the parking obstacle information around the parking space in a complex scene according to the parking area image, that is, whether there are potential dangerous factors in the parking route.

[0069] Step 404 : According to the obstacle type in the parking obstacle information, query each candidate parking area picture stored in the target database to determine the second parking area picture.

[0070] The type of obstacles around the second parking space in the second parking area picture is the obstacle type.

[0071] Step 405: Generate a plurality of training samples according to the second parking area image and the first parking area image.

[0072] Among them, the training samples are negative samples.

[0073] In the embodiment of the present application, in order to increase the number of training samples and improve the training effect of the initial parking space detection model, each candidate parking area picture stored in the target database can be queried according to the obstacle type in the parking obstacle information in the first parking area image. If the type of obstacles around the parking space in any candidate parking area picture is the obstacle type, then any candidate parking area picture can be used to determine the sequence sample, wherein any candidate parking area picture is multiple, which are called second candidate parking area pictures for identification.

[0074] Then, determine a first annotation result of the first parking space in the first parking area image and a second annotation result of the second parking space in the second parking area image, wherein the first annotation result indicates that the first parking space cannot be parked in; and the second annotation result indicates that the second parking space cannot be parked in. The annotated second parking area image and the first parking area image are used as multiple training samples. The multiple training samples are all negative samples. For each training sample, the initial parking space detection model is trained using the negative sample, so that the trained parking space detection model can accurately identify that the first parking space is vacant, but in fact the first parking space does not have the conditions for parking, based on the obstacles that affect parking around the parking space in the scene during the image recognition process. For example, there are obstacles on the parking path to the parking space, such as low tree stumps. Figure 5 A schematic diagram of a parking area image provided in an embodiment of the present application, such as Figure 5As shown, a camera installed on a vehicle, such as a fisheye camera, collects an image of the parking area. There is a tree stump B on the path leading to parking space A. The vehicle must pass through the tree stump B to park in parking space A. The tree stump B may damage the tire or chassis of the vehicle. Therefore, although parking space A is an empty parking space, due to the obstacle on the parking path, the vehicle must pass through the tree stump B to drive to parking space A, so that parking space A does not meet the parking conditions, that is, parking space A cannot be parked. Before the parking space detection model is trained using the parking area image of the scene, the initial parking space detection model cannot accurately identify this long-tail scene, that is, it cannot identify the obstacles in this scene and mistakenly believes that parking space A is a parking space that can be parked. After training with the image of the scene, the trained parking space detection model can recognize that there are obstacles in the parking area image and determine that there is no parking space that can be parked.

[0075] Step 406: Use the initial parking space detection model to detect the training samples to obtain prediction results.

[0076] The prediction result includes whether there is a parking space available in the training sample.

[0077] Step 407, determining a loss function according to the difference between the prediction result and the annotation result.

[0078] Step 408: adjusting the parameters of the initial parking space detection model according to the loss function to obtain a trained parking space detection model.

[0079] In an embodiment of the present application, business screening of autonomous driving parking data is achieved by identifying based on a visual detection model set in a server to obtain training samples of feature scenarios in which parking space detection cannot accurately identify parking space conditions. An initial parking space detection model set in a vehicle is trained based on the training samples to obtain a prediction result. A loss function is determined based on the difference between whether there is a parking space that can be parked determined in the prediction result and whether there is no parking space that can be parked marked in the marking result. The parameters of the initial parking space detection model are adjusted through the loss function to obtain a trained parking space detection model. Through training or updating training, the trained parking space detection model has the ability to recognize special obstacle scenarios, can accurately identify whether there is actually a parking space that can be parked, thereby improving the accuracy of parking space recognition, and further improving the safety of parking.

[0080] In the sample generation method of the embodiment of the present application, the visual detection model set in the server is used to identify the first parking area image. If parking obstacle information around the first parking space is identified, it is considered that the first parking space that the vehicle can park in according to the parking space detection model on the first parking area image is incorrect, and the first parking space is actually a parking space that cannot be parked in. Based on the first parking area image, a training sample of the parking space detection model is generated, and complex scenes in which vehicle detection is inaccurate are determined by the visual detection model set in the server. The images of the scene are used to train the initial parking space detection model, which can improve the diversity and accuracy of the samples, thereby improving the recognition accuracy of the trained parking space detection model and improving parking safety.

[0081] Based on the above embodiment, in an implementation of the embodiment of the present application, in response to the identification result that the parking obstacle information around the first parking space is identified, when the first parking space is determined to be a non-parking space, parking warning information is generated, wherein the parking warning information is used to prompt that there are obstacles on the parking path and / or in the parking space corresponding to the first parking space, and the parking space cannot be parked, so as to inform the owner of the reason why the parking space cannot be parked, and to prompt the user of the safety risk. The obstacles include tree stumps, special-shaped objects, potholes or suspended obstacles, etc.

[0082] In order to implement the above embodiment, the embodiment of the present application also proposes a parking space detection device.

[0083] Figure 6 A schematic diagram of the structure of a parking space detection device provided in an embodiment of the present application.

[0084] like Figure 6 As shown, the device may include:

[0085] The acquisition module 61 is used to acquire a first parking area image to be detected.

[0086] The determination module 62 is configured to use a parking space detection model to identify the parking spaces in the first parking area image, and determine whether there is a first parking space that can be parked; wherein the training samples used for training the parking space detection model are obtained by using a visual detection model in a server to identify a second parking area image collected by the vehicle, and in response to identifying the presence of parking obstacle information around the second parking space in the second parking area image, the determination module 62 is generated according to the second parking area image, wherein the second parking space is a parking space that can be parked that is obtained by the vehicle identifying the second parking area image based on the initial parking space detection model.

[0087] Further, in an implementation of the embodiment of the present application, there are multiple first parking spaces, and the determination module 62 is further configured to:

[0088] In response to the existence of a plurality of first parking spaces, sending the first parking area image to the server;

[0089] Acquire parking decision information of each first parking space sent by the server; wherein the parking decision information is used to indicate whether there are obstacles around the first parking space;

[0090] Determine, based on the parking decision information of each of the first parking spaces, a first candidate parking space with no surrounding obstacles;

[0091] A first target parking space to be parked is determined based on the first candidate parking space.

[0092] In an implementation of the embodiment of the present application, there are multiple first candidate parking spaces, and the determination module 62 is further configured to:

[0093] Determine, according to the sizes of the plurality of first candidate parking spaces, a second candidate parking space whose size matches the size of the vehicle;

[0094] The second candidate parking space is used as the first target parking space, and the vehicle is controlled to park in the first target parking space.

[0095] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and will not be repeated here.

[0096] In the parking space detection device of the embodiment of the present application, the long-tail data of complex scenes is screened through the visual detection model set in the server to determine the training samples for training the parking space detection model, thereby improving the diversity and accuracy of the training samples, and further improving the recognition accuracy of the trained parking space detection model, so that parking space recognition is performed based on the parking space detection model, thereby improving the recognition accuracy.

[0097] In order to implement the above embodiment, the embodiment of the present application also proposes a sample generating device.

[0098] Figure 7 A schematic diagram of the structure of a sample generation device provided in an embodiment of the present application.

[0099] like Figure 7 As shown, the device may include:

[0100] The acquisition module 71 is used to acquire a first parking area image collected by the vehicle; wherein the first parking area image includes a first parking space that can be parked and is obtained by the vehicle being identified based on an initial parking space detection model.

[0101] The recognition module 72 is used to recognize the first parking area image by using a visual detection model.

[0102] The generating module 73 is configured to generate a training sample of the initial parking space detection model according to the first parking area image in response to identifying the presence of parking obstacle information around the first parking space.

[0103] Furthermore, in an implementation of the embodiment of the present application, the visual detection model is a visual large language model, and the recognition module 72 is further used to:

[0104] Get the setting prompt words of the parking scene;

[0105] According to the set prompt word, the visual large language model is used to recognize the first parking area image to obtain parking obstacle information around the first parking space; wherein the parking obstacle information around the first parking space includes parking obstacle information on the first parking space and parking obstacle information on the parking path of the target vehicle from the current position into the first parking space.

[0106] In an implementation of the embodiment of the present application, there are multiple training samples, and the generating module 73 is further used to:

[0107] According to the obstacle type in the parking obstacle information, query each candidate parking area picture stored in the target database to determine the second parking area picture; wherein the type of obstacles around the second parking space in the second parking area picture is the obstacle type;

[0108] The plurality of training samples are generated according to the second parking area image and the first parking area image; wherein the training samples are negative samples.

[0109] In one implementation of the embodiment of the present application, the generating module 73 is further configured to:

[0110] Determine a first marking result of a first parking space in the first parking area image and a second marking result of a second parking space in the second parking area image; wherein the first marking result indicates that the first parking space cannot be parked in; and the second marking result indicates that the second parking space cannot be parked in;

[0111] The labeled second parking area image and the first parking area image are used as the multiple training samples.

[0112] In an implementation of the embodiment of the present application, the device further includes: a first training module, which is used to:

[0113] For each training sample, the initial parking space detection model is used to detect the training sample to obtain a prediction result; wherein the prediction result includes whether there is a parking space available in the training sample;

[0114] Determining a loss function based on the difference between the prediction result and the annotation result;

[0115] According to the loss function, the parameters of the initial parking space detection model are adjusted to obtain a trained parking space detection model.

[0116] In an implementation of the embodiment of the present application, the device further includes: a second training module, which is used to:

[0117] Acquire a parking area sample image; wherein the parking area sample image is annotated with parking obstacle information around the parking space;

[0118] Inputting the parking area sample image into an initial visual detection model to obtain predicted parking obstacle information of the parking space;

[0119] determining a loss function according to a difference between the predicted parking obstacle information and the marked parking obstacle information;

[0120] The initial visual detection model is trained according to the loss function to obtain the trained visual detection model.

[0121] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and will not be repeated here.

[0122] In the sample generation device of the embodiment of the present application, the visual detection model set in the server is used to identify the first parking area image. If the parking obstacle information around the first parking space is identified, it is considered that the first parking space that the vehicle can park in according to the initial parking space detection model is incorrect, and the first parking space is actually a parking space that cannot be parked in. According to the first parking area image, a training sample of the initial parking space detection model is generated, and the complex scene in which the vehicle detection is inaccurate is determined by the visual detection model set in the server. The image of the scene is used to train the initial parking space detection model, which can improve the diversity and accuracy of the samples, thereby improving the recognition accuracy of the trained parking space detection model and improving parking safety.

[0123] In order to implement the above embodiments, the present application also proposes a vehicle, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the above method embodiments is implemented.

[0124] In order to implement the above embodiments, the present application also proposes a non-temporary computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in the above method embodiments is implemented.

[0125] In order to implement the above embodiments, the present application also proposes a computer program product on which a computer program is stored. When the computer program is executed by a processor, the method described in the above method embodiments is implemented.

[0126] Figure 8 600 is a schematic diagram of a vehicle structure proposed in an embodiment of the present application. For example, vehicle 600 may be a hybrid vehicle, or a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicles. Vehicle 600 may be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.

[0127] Reference Figure 8 , the vehicle 600 may include various subsystems, for example, an infotainment system 610, a perception system 620, a decision control system 630, a drive system 640, and a computing platform 650. The vehicle 600 may also include more or fewer subsystems, and each subsystem may include multiple components. In addition, each subsystem and each component of the vehicle 600 may be interconnected by wire or wireless means.

[0128] In some embodiments, the infotainment system 610 may include a communication system, an entertainment system, and a navigation system, among others.

[0129] The perception system 620 may include several sensors for sensing information about the environment around the vehicle 600. For example, the perception system 620 may include a global positioning system (the global positioning system may be a GPS system, or a Beidou system or other positioning systems), an inertial measurement unit (IMU), a laser radar, a millimeter wave radar, an ultrasonic radar, and a camera.

[0130] The decision control system 630 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

[0131] The drive system 640 may include components that provide powered motion for the vehicle 600. In one embodiment, the drive system 640 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of an internal combustion engine, an electric motor, and an air compression engine. The engine is capable of converting energy provided by the energy source into mechanical energy.

[0132] Some or all functions of the vehicle 600 are controlled by a computing platform 650. The computing platform 650 may include at least one processor 651 and a memory 652, and the processor 651 may execute instructions 653 stored in the memory 652.

[0133] The processor 651 may be any conventional processor, such as a commercially available CPU. The processor may also include a graphics processor (Graphic Process Unit, GPU), a field programmable gate array (Field Programmable Gate Array, FPGA), a system on chip (System on Chip, SOC), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC) or a combination thereof.

[0134] The memory 652 may be implemented by any type of volatile or nonvolatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0135] In addition to the instructions 653 , the memory 652 may also store data, such as road maps, route information, and data such as the location, direction, and speed of the vehicle. The data stored in the memory 652 may be used by the computing platform 650 .

[0136] In the embodiment of the present disclosure, the processor 651 may execute instruction 653 to complete all or part of the steps of the above method embodiment.

[0137] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0138] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0139] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0140] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0141] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0142] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0143] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0144] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A parking space detection method, characterized in that: include: Acquire a first parking area image to be detected; A parking space detection model is used to identify the parking spaces in the first parking area image to determine whether there is a first parking space that can be parked; wherein, the training samples used for training the parking space detection model are obtained by using a visual detection model in a server to identify a second parking area image collected by the vehicle, and in response to identifying the presence of parking obstacle information around the second parking space in the second parking area image, the second parking space is a parking space that can be parked that is obtained by the vehicle identifying the second parking area image based on the initial parking space detection model.

2. The method according to claim 1, characterized in that There are multiple first parking spaces, and the method further includes: In response to the existence of a plurality of first parking spaces, sending the first parking area image to the server; Acquire parking decision information of each first parking space sent by the server; wherein the parking decision information is used to indicate whether there are obstacles around the first parking space; Determine, based on the parking decision information of each of the first parking spaces, a first candidate parking space with no surrounding obstacles; A first target parking space to be parked is determined based on the first candidate parking space.

3. The method according to claim 2, characterized in that There are multiple first candidate parking spaces, and determining a first target parking space to be parked according to the first candidate parking spaces includes: Determine, according to the sizes of the plurality of first candidate parking spaces, a second candidate parking space whose size matches the size of the vehicle; The second candidate parking space is used as the first target parking space, and the vehicle is controlled to park in the first target parking space.

4. A sample generation method, characterized in that: include: Acquire a first parking area image collected by the vehicle; wherein the first parking area image includes a first parking space that the vehicle can park in, which is identified based on an initial parking space detection model; Using a visual detection model to identify the first parking area image; In response to identifying the presence of parking obstacle information around the first parking space, a training sample of the initial parking space detection model is generated according to the first parking area image.

5. The method according to claim 4, characterized in that The visual detection model is a visual large language model, and the step of using the visual detection model to recognize the first parking area image includes: Get the setting prompt words of the parking scene; According to the set prompt word, the visual large language model is used to recognize the first parking area image to obtain parking obstacle information around the first parking space; wherein the parking obstacle information around the first parking space includes parking obstacle information on the first parking space and parking obstacle information on the parking path of the target vehicle from the current position into the first parking space.

6. The method according to claim 4, characterized in that The training samples are multiple, and the training samples of the initial parking space detection model are generated according to the first parking area image, including: According to the obstacle type in the parking obstacle information, query each candidate parking area picture stored in the target database to determine the second parking area picture; wherein the type of obstacles around the second parking space in the second parking area picture is the obstacle type; The plurality of training samples are generated according to the second parking area image and the first parking area image; wherein the training samples are negative samples.

7. The method according to claim 6, characterized in that The step of generating the plurality of training samples according to the second parking area image and the first parking area image comprises: Determine a first marking result of a first parking space in the first parking area image and a second marking result of a second parking space in the second parking area image; wherein the first marking result indicates that the first parking space cannot be parked in; and the second marking result indicates that the second parking space cannot be parked in; The labeled second parking area image and the first parking area image are used as the multiple training samples.

8. The method according to claim 7, characterized in that The method further comprises: For each training sample, the initial parking space detection model is used to detect the training sample to obtain a prediction result; wherein the prediction result includes whether there is a parking space available in the training sample; Determining a loss function based on the difference between the prediction result and the annotation result; According to the loss function, the parameters of the initial parking space detection model are adjusted to obtain a trained parking space detection model.

9. The method according to any one of claims 4 to 8, characterized in that: The method further comprises: Acquire a parking area sample image; wherein the parking area sample image is annotated with parking obstacle information around the parking space; Inputting the parking area sample image into an initial visual detection model to obtain predicted parking obstacle information of the parking space; determining a loss function according to a difference between the predicted parking obstacle information and the marked parking obstacle information; The initial visual detection model is trained according to the loss function to obtain the trained visual detection model.

10. A parking space detection device, characterized in that: include: An acquisition module, used for acquiring an image of a first parking area to be detected; a determination module, configured to identify the parking spaces in the first parking area image by using a parking space detection model, and determine whether there is a first parking space available for parking; wherein the training samples used for training the parking space detection model are obtained by identifying a second parking area image collected by the vehicle by using a visual detection model in a server, and in response to identifying the presence of parking obstacle information around the second parking space in the second parking area image, the determination module is generated according to the second parking area image, wherein the second parking space is a parking space available for parking obtained by the vehicle through the recognition of the second parking area image based on the initial parking space detection model.

11. A sample generating device, characterized in that: include: An acquisition module, configured to acquire a first parking area image collected by the vehicle; wherein the first parking area image includes a first parking space that the vehicle can park in, which is identified based on an initial parking space detection model; A recognition module, configured to recognize the first parking area image using a visual detection model; A generating module is used to generate a training sample of the initial parking space detection model according to the first parking area image in response to identifying the parking obstacle information around the first parking space.

12. A vehicle, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Implement the method according to any one of claims 1 to 3, or implement the method according to any one of claims 4 to 9.

13. A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a mobile terminal, the mobile terminal is enabled to execute the method according to any one of claims 1 to 3, or implement the method according to any one of claims 4 to 9.

14. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 3, or implements the method according to any one of claims 4 to 9.

Images