Vehicle attitude detection method and device and electronic equipment

Abstract

The embodiment of the invention discloses a vehicle attitude detection method and device and electronic equipment, and the method comprises the steps: obtaining a target image of a target vehicle in a parking verification state, carrying out the image recognition of the target image, obtaining an object recognition result which comprises the position information of the target vehicle and a reference object, and carrying out the recognition of the target image; and determining attitude information of the target vehicle according to an object recognition result, and determining that the target vehicle is parked in a standard manner in response to the attitude information representing a standard attitude. Therefore, according to the embodiment of the invention, the method can achieve the recognition and analysis of the features of the image, improves the judgment accuracy of the vehicle attitude, reduces the dependence of the judgment of the vehicle attitude on a sensor and other hardware, and reduces the maintenance cost.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and more specifically, to a vehicle posture detection method, device and electronic equipment. Background Art

[0002] With the popularity of shared vehicles (such as two-wheeled vehicles, etc.), the standardized management of a large number of shared vehicles has become an issue that cannot be ignored, among which the parking problem of shared vehicles is particularly prominent. At present, some users do not park their vehicles in designated areas or dump them after use, which makes the parking of shared vehicles occupy more space and affects the overall parking aesthetics of the street. At the same time, it may cause road congestion and other phenomena. Therefore, it is crucial to perform vehicle posture detection and verification when parking the vehicle to achieve standardized parking. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides a vehicle posture detection method, device and electronic device, which improves the accuracy of vehicle posture judgment by identifying and analyzing image features, reduces the dependence of vehicle posture judgment on hardware such as sensors, and reduces maintenance costs.

[0004] In a first aspect, an embodiment of the present invention provides a vehicle posture detection method, the method comprising:

[0005] Acquire a target image, wherein the target image includes a target vehicle in a parking verification state;

[0006] Performing image recognition on the target image to obtain an object recognition result, wherein the object recognition result at least includes position information of the target vehicle and a reference object;

[0007] Determining the posture information of the target vehicle according to the object recognition result;

[0008] In response to the posture information representing a standard posture, it is determined that the target vehicle is parked in a standard manner.

[0009] Optionally, performing image recognition on the target image to obtain an object recognition result includes:

[0010] The target image is input into a pre-trained object recognition model for image processing to obtain the object recognition result.

[0011] Optionally, the object recognition model is an image segmentation model, and the image segmentation model includes an image coding layer and a convolution layer;

[0012] The inputting the target image into a pre-trained object recognition model for image processing to obtain the object recognition result comprises:

[0013] Performing image data processing on the target image through the image coding layer to obtain image feature information;

[0014] The image feature information is processed through the convolution layer to obtain the object recognition result.

[0015] Optionally, the reference object is a combination of one or more of the following: a road edge sign, a parking area edge line, and at least one parked vehicle other than the target vehicle.

[0016] Optionally, determining the posture information of the target vehicle according to the object recognition result includes:

[0017] The target image and the object recognition result are input into a pre-trained posture recognition model for processing to obtain the posture information of the target vehicle.

[0018] Optionally, the method further includes:

[0019] In response to the driving data corresponding to the target vehicle indicating that the initial posture of the target vehicle is a non-standard posture, image acquisition prompt information is generated.

[0020] Optionally, the method is applied to a user terminal, and the image acquisition prompt information includes prompt content and image acquisition controls;

[0021] The acquiring of the target image comprises:

[0022] In response to the image acquisition control being triggered, calling a camera in the user terminal;

[0023] In response to the shooting control being triggered, the target image is acquired.

[0024] Optionally, the method is applied to a server or a vehicle, and acquiring the target image includes:

[0025] Sending the image acquisition prompt information to the user terminal;

[0026] Receive the target image captured by the user terminal.

[0027] In a second aspect, an embodiment of the present invention provides a vehicle posture detection device, the device comprising:

[0028] An image acquisition unit is configured to acquire a target image, wherein the target image includes a target vehicle in a parking verification state;

[0029] an image recognition unit, configured to perform image recognition on the target image and obtain an object recognition result, wherein the object recognition result at least includes position information of the target vehicle and a reference object;

[0030] a posture detection unit, configured to determine the posture information of the target vehicle according to the object recognition result;

[0031] The parking management unit is configured to determine that the target vehicle is parked in a standardized manner in response to the posture information representing a standard posture.

[0032] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described above.

[0033] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0034] In a fifth aspect, an embodiment of the present invention provides a computer program product, which, when executed on a computer, enables the computer to execute the method described above.

[0035] The embodiment of the present invention obtains a target image of a target vehicle in a parking verification state, performs image recognition on the target image, obtains an object recognition result including the position information of the target vehicle and a reference object, determines the posture information of the target vehicle according to the object recognition result, and determines the standard parking of the target vehicle in response to the standard posture represented by the posture information. Therefore, the embodiment can improve the judgment accuracy of the vehicle posture by identifying and analyzing the features of the image, and reduce the dependence of the vehicle posture judgment on hardware such as sensors, thereby reducing the maintenance cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0037] Figure 1 is a flow chart of a vehicle posture detection method according to an embodiment of the present invention;

[0038] Figure 2-Figure 3 is a schematic diagram of a user terminal interface of an embodiment of the present invention;

[0039] Figure 4 is a schematic diagram of an object recognition process according to an embodiment of the present invention;

[0040] Figure 5 is a schematic diagram of a gesture recognition process according to an embodiment of the present invention;

[0041] Figure 6 is a schematic diagram of a vehicle posture detection system according to an embodiment of the present invention;

[0042] Figure 7 is an interactive flow chart of a vehicle posture detection system according to an embodiment of the present invention;

[0043] Figure 8 is an interaction flow chart of another vehicle posture detection system according to an embodiment of the present invention;

[0044] Fig. 9 is a schematic diagram of a vehicle posture detection device according to an embodiment of the present invention;

[0045] Fig.10 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The present application is described below based on embodiments, but the present application is not limited to these embodiments. In the detailed description of the present application below, some specific details are described in detail. It is possible for those skilled in the art to fully understand the present application without the description of these details. In order to avoid confusing the essence of the present application, known methods, processes, flows, components and circuits are not described in detail.

[0047] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.

[0048] Unless the context clearly requires otherwise, the words "include", "comprising" and similar words throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, the meaning is "including but not limited to".

[0049] In the description of this application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" is two or more.

[0050] The solutions described in this specification and in the examples, if they involve the processing of personal information, will be processed on the premise of having a legal basis (such as obtaining the consent of the subject of personal information, or being necessary for the performance of a contract, etc.), and will only be processed within the scope of regulations or agreements. If a user refuses to process personal information other than the necessary information for basic functions, it will not affect the user's use of basic functions.

[0051] In the relevant technical solutions, the vehicle posture verification when the user returns the car mainly relies on the sensor equipment (such as gyroscope, etc.) in the vehicle hardware. The use of a single sensor device may result in low accuracy in the identification of the vehicle posture, which may lead to the situation that even if the vehicle is parked in a standardized manner, it is verified as non-standard, which makes the user experience poor. If multiple sensor devices are used to verify the vehicle posture, it is necessary to increase the hardware cost of the vehicle, and the hardware is easy to damage, the maintenance cost is high, and it is difficult to adapt the relevant functions to the deployed vehicles. At the same time, the coordination between sensors may further affect the verification accuracy of the vehicle posture. Therefore, this embodiment provides a vehicle posture detection method, device and electronic device, which improves the judgment accuracy of the vehicle posture by identifying and analyzing the features of the image containing the parked vehicle, and reduces the dependence of the vehicle posture judgment on hardware such as sensors, thereby reducing the maintenance cost.

[0052] Figure 1 It is a flow chart of the vehicle posture detection method of an embodiment of the present invention. The vehicle posture detection method of this embodiment can be applied to a user terminal, that is, after the user parks, a target image containing a target vehicle is photographed through the user terminal, and a vehicle posture detection is performed based on the target image to determine whether the parking is standardized, and the target vehicle is controlled to be locked when the vehicle posture is standardized. In another optional implementation, the vehicle posture detection method of this embodiment can also be applied to a central control device of a server or a vehicle. After the user finishes riding, a target image containing a target vehicle is photographed through the user terminal, and the target image is sent to the central control device of the server or the target vehicle. The central control device of the server or the target vehicle performs vehicle posture detection based on the target image to determine whether the parking is standardized, and the target vehicle is controlled to be locked when the vehicle posture is standardized. In other words, the vehicle posture detection method of this embodiment can be deployed at any end involved in parking management, and this embodiment does not limit this.

[0053] like Figure 1 As shown, the vehicle posture detection method of this embodiment includes the following steps:

[0054] Step S110, acquiring a target image. The target image includes a target vehicle in a parking verification state. In this embodiment, the target image including the target vehicle is captured by a user terminal. Furthermore, if the execution subject is a server or a vehicle, the target image is acquired by communicating with the user terminal.

[0055] In an optional implementation, the user terminal, the server and the vehicle communicate with each other via wireless communication. Furthermore, the user terminal can communicate with the vehicle via the server or via short-range communication (such as Bluetooth communication), which is not limited in this embodiment.

[0056] In an optional implementation, in response to detecting the user's parking behavior, the initial posture of the target vehicle is first determined based on the data collected by the sensors (such as inertial navigation unit IMU, gyroscope, acceleration sensor, etc.) set on the target vehicle, and it is determined whether the initial posture is a standard posture. When the initial posture of the target vehicle is a standard posture, it is determined that the target vehicle has achieved standardized parking and the target vehicle is locked. When the initial posture of the target vehicle is a non-standard posture, an image acquisition prompt message is generated. Optionally, the user's parking behavior can be triggered based on the user's request to end riding or lock. Furthermore, based on the difference in the lock body of the target vehicle, the user can generate a request to end riding or a request to lock through the user terminal, or can generate a request to end riding or a request to lock based on manual locking. This embodiment does not limit this.

[0057] Optionally, the posture of the target vehicle may include whether the vehicle is in the parking area, whether it is dumped, and whether it is parked at a predetermined angle. Parking at a predetermined angle means that the angle between the vehicle and the edge of the road or the edge of the parking area is within a predetermined angle range, and the predetermined angle range is used to characterize that the vehicle is perpendicular or substantially perpendicular to the edge of the road or the edge of the parking area. Therefore, the predetermined angle range may be [90°-a, 90°+a]. Wherein, a may be customized according to the actual parking area situation, for example, a may be 15°, etc., and the embodiment is not limited to this.

[0058] Furthermore, this embodiment can determine the initial direction according to the positioning data during the vehicle driving process, determine the relative motion information of the target vehicle according to the driving data collected by the sensor device, and determine the parking direction according to the initial direction and the relative motion information. Thus, the accuracy of identifying the parking direction can be improved.

[0059] Furthermore, the present embodiment can also determine whether the vehicle has tipped over according to the angle between the three-axis acceleration data in the driving data collected by the sensor device and the gravity acceleration. When the vehicle is upright, the downward acceleration in the three-axis acceleration is basically consistent with the direction of the gravity acceleration, and when the vehicle is tipped over, its downward acceleration is obliquely downward and has a certain angle with the direction of the gravity acceleration. Therefore, the present embodiment can determine whether the vehicle has tipped over by determining the size of the angle.

[0060] Therefore, the target vehicle can determine the initial posture according to the driving data collected by the sensor device, and control the locking or generate image acquisition prompt information based on whether the initial posture meets the conditions (that is, whether it is a standard posture), and send the image acquisition prompt information to the user terminal, or send the image acquisition prompt information to the user terminal through the server.

[0061] In another optional implementation, the target vehicle can send the driving data collected by the sensor device to the server in real time (or after the parking behavior is detected), and the server determines the initial posture based on the driving data, and based on whether the initial posture meets the conditions, sends a locking instruction to the target vehicle to control the locking of the target vehicle, or generates image acquisition prompt information and sends the image acquisition prompt information to the user terminal. Further optionally, the target vehicle can also upload the initial posture determined based on the driving data collected by the sensor device to the server after detecting the parking behavior of the user, so that the server sends a locking instruction to the target vehicle based on whether the initial posture meets the conditions to control the locking of the target vehicle, or generates image acquisition prompt information and sends the image acquisition prompt information to the user terminal.

[0062] In other optional implementations, the initial posture of the target vehicle may also be determined and image acquisition prompt information may be generated by the user terminal, and this embodiment does not limit this.

[0063] In an optional implementation, the image acquisition prompt information may include prompt content and image acquisition controls. Optionally, the prompt content may include information such as the current vehicle posture verification status, image acquisition prompts, and / or vehicle posture adjustment prompts. For example, "The vehicle is not currently parked in a standard manner, please adjust the vehicle posture. If it has been adjusted, please take a photo of the vehicle for posture verification." It should be understood that this embodiment does not limit the specific prompt information.

[0064] Figure 2-3 Schematic diagram of the user terminal interface of the embodiment of the present invention. This embodiment takes the return of the vehicle through the user terminal as an example. Figure 2 and Figure 3 As shown, in response to the confirmation vehicle return control 21 displayed in the user terminal interface being triggered, the vehicle return verification process is entered. In response to the driving data corresponding to the target vehicle indicating that the initial posture of the target vehicle is a non-standard posture, the user terminal interface is controlled to display image acquisition prompt information 3.

[0065] The image acquisition prompt information 3 includes prompt content and image acquisition controls 31. The prompt content may be "the vehicle is not currently parked in a standard manner, please adjust the vehicle posture, if it has been adjusted, please take a photo of the vehicle for posture verification".

[0066] In this embodiment, in response to the image acquisition control 31 being triggered, the camera in the user terminal is called, and in response to the shooting control being triggered, the target image is acquired. It should be understood that the relevant application of this embodiment obtains the permission to call the camera from the user in advance, or prompts the user to enable the camera calling permission when the image acquisition control 31 is triggered and the camera permission is not obtained, and calls the camera in the user terminal after obtaining the permission.

[0067] Further optional, Figure 3 The user terminal interface shown may also include a repair control, so that when the user actually parks the car properly but the initial posture is judged as improper parking, the user can return the car by triggering the repair control, thereby further improving the user experience.

[0068] Step S120, performing image recognition on the target image to obtain an object recognition result. The object recognition result includes at least the position information of the target vehicle and the reference object. The position information of the target vehicle and the reference object includes the position of the target vehicle in the target image and the position of the reference object in the target image.

[0069] Optionally, the reference object is a combination of one or more of the following: a road edge mark, a parking area edge line, and at least one parked vehicle other than the target vehicle. Among them, the parking area edge line is parallel to the road edge. The parking posture of the vehicle is generally determined to be properly parked when it is perpendicular to the parking area edge line (and / or the road edge). In other optional modes, if the target vehicle is parallel or substantially parallel to other surrounding vehicles, it can also be determined that the target vehicle is properly parked. Therefore, this embodiment can determine the parking posture of the target vehicle with reference to the road edge mark, the parking area edge line, and / or at least one parked vehicle other than the target vehicle, so as to further improve the accuracy of judging the parking posture of the vehicle.

[0070] In an optional implementation, this embodiment inputs the target image into a pre-trained object recognition model for image processing to obtain an object recognition result, so as to improve the accuracy of target detection and further improve the accuracy of judging the parking posture of the vehicle. Optionally, the object recognition model is an image segmentation model. The image segmentation model is a model that divides an image into multiple segments and maps each pixel in the image to an object with a category label.

[0071] Further optionally, the image segmentation model of this embodiment includes an image encoding layer (Image Encoder) and a convolution layer. Among them, this embodiment performs image data processing on the target image through the image encoding layer to obtain image feature information, and processes the image feature information through the convolution layer to obtain object recognition results. Optionally, the image encoding layer of this embodiment adopts the Vision Transformer architecture to perform data processing on the input target image.

[0072] It should be understood that this embodiment does not limit the type of object recognition model, as long as it can achieve target detection. When using an image segmentation model, it does not limit the architecture of the image segmentation model, as long as it can achieve target detection using the principle of image segmentation.

[0073] In an optional implementation, the present embodiment trains the object recognition model through a pre-acquired first training set. The first training set may include photographed images of various types of vehicles in parking postures, so that the trained object recognition model can accurately identify the vehicle and reference object in the target image. Optionally, the images in the first training set include label information of the target vehicle and various types of reference objects. The reference object may be one or more of a road edge sign, a parking area edge line, and other parked vehicles, and the present embodiment does not limit this.

[0074] In another optional implementation, if the training samples of the parking posture of the vehicle are insufficient, the object recognition model may be pre-trained using images in other application scenarios, and then the pre-trained object recognition model may be trained using a first training set of captured images of the parking postures of various types of vehicles to perform personalized parameter adjustment and obtain a trained object recognition model. This embodiment does not limit the training process of the object recognition model.

[0075] Furthermore, since the user usually takes pictures next to the target vehicle, the target vehicle is positioned closer to the center and has a larger area in the target image, which can be used to distinguish the target vehicle from other parked vehicles.

[0076] Figure 4 FIG. 1 is a schematic diagram of an object recognition process according to an embodiment of the present invention. Figure 4 As shown, image 4 is a target image captured by a user terminal. Image 4 is input into an object recognition model 40 for target detection processing, and a target vehicle 41, a parking area edge line 42, a road edge 43, and other parked vehicles 44 can be detected. Therefore, this embodiment can determine the posture information of the target vehicle according to the positional relationship between the target vehicle 41 and at least one reference object.

[0077] Step S130, determining the posture information of the target vehicle according to the object recognition result. In an optional implementation, this embodiment inputs the target image and the object recognition result into a pre-trained posture recognition model for processing to obtain the posture information of the target vehicle.

[0078] In an optional implementation, the present embodiment trains the posture recognition model through a pre-acquired second training set. The second training set includes a plurality of image samples containing various types of vehicle postures and target image data and reference object data in the image samples. The target image data at least includes the position information of the target vehicle in the corresponding image sample, and the reference object data at least includes the position information of the reference object in the corresponding image sample. Furthermore, the image sample also includes a vehicle posture label, such as a standard posture, a fallen posture, a tilted posture, etc. Further optionally, the images in the second training set and the first training set may be the same or different.

[0079] In an optional implementation, the target image data and reference object data in the image samples in the second training set may be manually annotated, or may be the output of the corresponding image samples processed by a pre-processed object recognition model, and this embodiment is not limited to this.

[0080] In an optional implementation, the posture recognition model of this embodiment may be a deep learning network. It should be understood that this embodiment does not limit the network type of the posture recognition model.

[0081] In an optional implementation, the object recognition model and the gesture recognition model of this embodiment are independently trained based on the corresponding first training set and second training set. In other optional implementations, the object recognition model and the gesture recognition model can also be jointly trained, and this embodiment does not limit this.

[0082] Figure 5 FIG. 1 is a schematic diagram of a gesture recognition process according to an embodiment of the present invention. Figure 5 As shown, image 5 is a target image captured by a user terminal, and image 5 is input into an object recognition model 50 for target detection processing to obtain an object recognition result 51. The object recognition result 51 includes the position information of the target vehicle and at least one reference object in image 5. Furthermore, in this embodiment, image 5 and object recognition result 51 are input into a posture recognition model 52 for processing to obtain the posture information of the target vehicle.

[0083] Take the reference object as the edge of the parking area as an example. Figure 5As shown, in this embodiment, the object recognition model 50 can detect the position information of the target vehicle and the edge line of the parking area in the image 5, and the posture recognition model 52 can detect the positional relationship between the target vehicle and the edge line of the parking area through the image 5, the target vehicle and the position information of the edge line of the parking area in the image 5 to determine the posture information of the target vehicle. For example, the size of the angle α between the parking direction of the target vehicle and the edge line of the parking area can be detected, and whether the parking direction of the target vehicle is within a predetermined angle range (for example, 80°-110°) is determined based on the size of the angle α. If the size of the angle α is less than 80°, it is determined that the target vehicle is in an inclined parking posture.

[0084] In other optional implementations, if other parked vehicles in the parking area are used as reference objects, it can be determined whether the parking direction of the target vehicle is parallel or substantially parallel to the parking directions of other parked vehicles, that is, whether the angle between the parking direction of the target vehicle and the parking directions of other parked vehicles is less than an angle threshold (e.g., 10°, etc.). It should be understood that in this embodiment, the parking direction of the vehicle is the direction of the line from the front to the rear of the vehicle.

[0085] In another optional implementation, since the shooting angle of the target vehicle has a certain influence on the recognition result of the vehicle posture, this embodiment can improve the accuracy of the object recognition model and the posture recognition model by shooting the vehicle in various postures based on various shooting angles during the model training process, and can also refer to the display form of other parked vehicles other than the target vehicle in the image during the posture recognition process to further improve the recognition accuracy of the vehicle posture.

[0086] Step S140 , in response to the posture information representing the standard posture, determining that the target vehicle is parked in a standardized manner.

[0087] In this embodiment, in response to the identified posture information of the target vehicle indicating that the target vehicle is in a standard posture, the target vehicle is determined to be parked in a standardized manner, the target vehicle is controlled to be locked, and the vehicle return process ends.

[0088] If the identified target vehicle's posture information is in a non-standard posture, such as a tilted parking posture, a dumped posture, etc., a reminder message is displayed or voice broadcasted through the user terminal or the target vehicle. Among them, the reminder message includes a prompt message prompting the user to adjust the vehicle's parking posture. Further, if the identified target vehicle's posture information is still in a non-standard posture, the control continues the vehicle return process. If the user determines that the parking has been standardized, the user can verify again by taking an image or end the replacement process through a repair process. This embodiment does not limit this.

[0089] In an optional implementation, before performing vehicle posture detection, the present embodiment may also first determine the position of the target vehicle to determine whether the target vehicle is located in the parking area. The position of the target vehicle may be determined by a positioning device provided on the target vehicle or a positioning device provided on the user terminal, and the present embodiment does not limit this.

[0090] This embodiment obtains a target image of a target vehicle in a parking verification state, performs image recognition on the target image, obtains an object recognition result including the position information of the target vehicle and a reference object, determines the posture information of the target vehicle according to the object recognition result, and determines the standard parking of the target vehicle in response to the standard posture represented by the posture information. Thus, this embodiment can improve the judgment accuracy of the vehicle posture by identifying and analyzing the features of the image, and reduce the dependence of the vehicle posture judgment on hardware such as sensors, thereby reducing the maintenance cost.

[0091] Figure 6 Schematic diagram of a vehicle posture detection system according to an embodiment of the present invention. This embodiment is described by taking a parking interaction system for a shared two-wheeled vehicle as an example. Figure 6 As shown, the vehicle posture detection system of this embodiment includes a vehicle 61, a server 62 and a user terminal 63. Among them, the vehicle 61, the server 62 and the user terminal 63 are connected in communication via wireless communication. Further, the vehicle 61 and the user terminal 63 can communicate wirelessly via short-range communication (such as Bluetooth communication), and this embodiment does not limit the communication method between the terminals.

[0092] The user terminal 63 uses the shared vehicle 61 by scanning a code to ride. During the riding process, the sensor equipment (such as an inertial navigation unit IMU, or a gyroscope, etc.) deployed on the vehicle 61 obtains driving data. Furthermore, a positioning device is also provided on the vehicle 61 to obtain the position information of the vehicle 61 in real time. In other optional implementations, the position of the vehicle 61 can also be determined by the positioning device provided in the user terminal 63.

[0093] Furthermore, after the vehicle return process is triggered, the vehicle 61 can determine whether the vehicle 61 is located in the parking area through the position determined by the positioning device and the position of the parking area in the area corresponding to the position, and determine the initial posture of the vehicle through the driving data collected by the sensor device. In other optional implementations, the vehicle 61 can also send the position and driving data to the server 62 or the user terminal 63, so that the server 62 or the user terminal 63 can determine whether the vehicle 61 is located in the parking area and the initial posture of the vehicle 61.

[0094] Furthermore, this embodiment can determine the parking direction of the vehicle 61 according to the initial direction of the vehicle 61 and the driving direction change information represented by the driving data.

[0095] Furthermore, the present embodiment can also determine whether the vehicle has tipped over by using the angle between the three-axis acceleration data in the driving data and the gravity acceleration. When the vehicle is upright, the downward acceleration in the three-axis acceleration is basically consistent with the direction of the gravity acceleration, while when the vehicle is tipping over, its downward acceleration is obliquely downward and has a certain angle with the direction of the gravity acceleration. Therefore, the present embodiment can determine whether the vehicle has tipped over by determining the size of the angle.

[0096] In this embodiment, if the initial posture of the vehicle 61 represents a standard posture, the vehicle 61 is controlled to be locked and the vehicle return process is terminated. If the initial posture of the vehicle 61 represents a non-standard posture, the user terminal 63 is controlled to display an image acquisition prompt message to prompt the user to further verify the posture of the vehicle 61 by taking a photo of the vehicle 61.

[0097] Furthermore, in this embodiment, the user terminal 63 (or the server 62, or the vehicle 61) performs image recognition on the captured vehicle image to obtain an object recognition result including the position information of the vehicle 61 and the reference object, and determines the posture information of the vehicle 61 according to the object recognition result. In this way, the accuracy of the vehicle posture verification can be further improved, and the situation where the user adjusts the vehicle posture multiple times but is still judged as a non-standard posture can be avoided, thereby improving the user experience.

[0098] This embodiment first determines the initial posture of the vehicle based on the driving data collected by the sensors installed on the vehicle. When the initial posture represents a non-standard posture, image acquisition prompt information is provided to prompt the user to further verify the posture of the vehicle by taking a photo of the vehicle. After acquiring the captured image, image recognition is performed on the captured vehicle image to obtain an object recognition result including the location information of the vehicle and the reference object, and the posture information of the vehicle is determined based on the object recognition result. In this way, the accuracy of the vehicle posture verification can be further improved, avoiding the situation where the user adjusts the vehicle posture multiple times and it is still judged as a non-standard posture, thereby improving the user experience. At the same time, this embodiment can achieve accurate posture verification without adding sensors to the vehicle, and can also adapt to the vehicles that have been deployed, reducing the dependence of vehicle posture judgment on hardware such as sensors, and reducing maintenance costs.

[0099] Figure 761 is an interactive flow chart of a vehicle posture detection system according to an embodiment of the present invention. This embodiment is described by taking the execution of the vehicle posture detection method on the server side as an example. It should be understood that this embodiment does not limit the interaction between the vehicle 61, the server 62 and the user terminal 63 to this, and other interactive methods that can realize posture detection can be applied to this embodiment. Among them, the vehicle 61 is provided with a positioning device and a sensor device.

[0100] like Figure 7 As shown, the interaction process of the vehicle posture detection system of this embodiment includes the following steps:

[0101] Step S1, during the use of the vehicle, the vehicle 61 periodically obtains position information through a positioning device and obtains driving data through a sensor device.

[0102] In step S2 , the vehicle 61 uploads the acquired location information and driving data to the server 62 .

[0103] Step S3, after using the vehicle, the user sends a request to confirm returning the vehicle to the server 62 via the user terminal 63.

[0104] Step S4, the server 62 determines the initial posture of the vehicle 61 according to the position information and driving data received in real time.

[0105] Step S5: the server 62 determines whether the initial posture is a standard posture.

[0106] Step S6 , the server 62 sends a message indicating that the vehicle is returned successfully to the user terminal 63 in response to the initial posture representing the standard posture.

[0107] In step S7 , the server 62 sends a vehicle lock instruction to the vehicle 61 in response to the initial posture representing the standard posture.

[0108] In step S8, the vehicle 61 is locked.

[0109] Among them, steps S6-S8 are steps executed to determine that the current posture of the target vehicle is the standard posture. In the posture determination process of the vehicle return process, in response to the vehicle posture being the standard posture, steps S6-S8 are executed to end the vehicle return process.

[0110] Step S9: the server 62 generates image acquisition prompt information in response to the initial posture characterizing the non-standard posture.

[0111] Step S10 : the server 62 sends image acquisition prompt information to the user terminal 63 .

[0112] Step S11, the user terminal 63 displays or broadcasts the prompt content in the image acquisition prompt information, and controls the terminal interface to display the shooting control.

[0113] Furthermore, the server 62 may also send the image acquisition prompt information to the vehicle 61 , so that the vehicle 61 broadcasts the prompt content in the image acquisition prompt information to further give the user a prompt.

[0114] Step S12: In response to the shooting control being triggered, the user terminal 63 calls the camera in the user terminal 63 to capture the target image.

[0115] Step S13 : the user terminal 63 uploads the target image to the server 62 .

[0116] In step S14, the server 62 inputs the target image into the object recognition model to obtain an object recognition result.

[0117] In step S15, the server 62 inputs the target image and the object recognition result into the posture recognition model to obtain the posture information of the vehicle 61. It should be understood that the specific process of performing posture recognition based on images in this embodiment is basically the same as that in the above embodiment, and will not be repeated here.

[0118] Furthermore, after step S15, it is determined whether the currently determined posture information is the standard posture. When it is determined to be the standard posture, steps S6-S8 are continued to be executed to end the vehicle return process.

[0119] In step S16, the server 62 provides a posture adjustment prompt in response to the posture information indicating the non-standard posture. Optionally, the server 62 may send the posture adjustment prompt information to the vehicle 61 or the user terminal 63 so that the posture adjustment prompt information is broadcast or displayed by the vehicle 61 or the user terminal 63.

[0120] This embodiment first determines the initial posture of the vehicle based on the driving data collected by the sensors installed on the vehicle. When the initial posture represents a non-standard posture, image acquisition prompt information is provided to prompt the user to further verify the posture of the vehicle by taking a photo of the vehicle. After acquiring the captured image, image recognition is performed on the captured vehicle image to obtain an object recognition result including the location information of the vehicle and the reference object, and the posture information of the vehicle is determined based on the object recognition result. In this way, the accuracy of the vehicle posture verification can be further improved, avoiding the situation where the user adjusts the vehicle posture multiple times and it is still judged as a non-standard posture, thereby improving the user experience. At the same time, this embodiment can achieve accurate posture verification without adding sensors to the vehicle, and can also adapt to the vehicles that have been deployed, reducing the dependence of vehicle posture judgment on hardware such as sensors, and reducing maintenance costs.

[0121] Figure 861 is an interactive flow chart of another vehicle posture detection system of an embodiment of the present invention. This embodiment is described by taking the execution of the vehicle posture detection method in the user terminal as an example. It should be understood that this embodiment does not limit the interaction between the vehicle 61, the server 62 and the user terminal 63 to this, and other interactive methods that can realize posture detection can be applied to this embodiment. Among them, the vehicle 61 is provided with a positioning device and a sensor device.

[0122] like Figure 8 As shown, the interaction process of the vehicle posture detection system of this embodiment includes the following steps:

[0123] Step S1 ′: during the use of the vehicle, the vehicle 61 periodically obtains position information through a positioning device and obtains driving data through a sensor device.

[0124] Step S2 ′: after using the vehicle, the user sends a request to confirm returning the vehicle to the server 62 via the user terminal 63 .

[0125] Step S3 ′: after receiving the vehicle return confirmation request, the server 62 sends a vehicle lock instruction to the vehicle 61 .

[0126] In step S4 ′, the vehicle 61 determines the initial posture of the vehicle 61 according to the position information and driving data received in real time.

[0127] In step S5', the vehicle 61 determines whether the initial posture is a standard posture.

[0128] In step S6 ′, the vehicle 61 is locked in response to the initial posture representing the standard posture.

[0129] Step S7 ′: after the vehicle 61 is locked, a locking success message is sent to the server 62 .

[0130] Step S8 ′: after receiving the locking success message, the server 62 sends a message of successful vehicle return to the user terminal 63 .

[0131] Among them, steps S6'-S8' are steps executed to determine that the current posture of the target vehicle is the standard posture. In the posture determination process of the vehicle return process, in response to the vehicle posture being the standard posture, steps S6'-S8' are executed to end the vehicle return process.

[0132] In step S9 ′, the vehicle 61 generates and broadcasts image acquisition prompt information in response to the initial posture representing a non-standard posture.

[0133] Step S10 ′: the vehicle 61 sends the image acquisition prompt information to the user terminal 63 through the server 62 .

[0134] Step S11 ′: the user terminal 63 displays or broadcasts the prompt content in the image acquisition prompt information, and controls the terminal interface to display the shooting control.

[0135] Step S12 ′: in response to the shooting control being triggered, the user terminal 63 calls the camera in the user terminal 63 to capture the target image.

[0136] In step S13 ′, the user terminal 63 inputs the target image into the object recognition model to obtain an object recognition result.

[0137] In step S14', the user terminal 63 inputs the target image and the object recognition result into the posture recognition model to obtain the posture information of the vehicle 61. It should be understood that the specific process of posture recognition based on the image in this embodiment is basically the same as that in the above embodiment, and will not be repeated here.

[0138] In step S15 ′, the user terminal 63 uploads the posture information of the vehicle 61 to the server 62 .

[0139] In step S16 ′, the server 62 provides a posture adjustment prompt in response to the posture information representing a non-standard posture, and sends a vehicle lock instruction to the vehicle 61 in response to the posture information representing a standard posture.

[0140] Optionally, the server 62 may send the posture adjustment prompt information to the vehicle 61 or the user terminal 63, so as to broadcast or display the posture adjustment prompt information through the vehicle 61 or the user terminal 63. Further, after receiving the posture information representing the standard posture and the vehicle locking instruction from the server 62, the vehicle 61 executes steps S6'-S8' to end the vehicle return process.

[0141] This embodiment first determines the initial posture of the vehicle based on the driving data collected by the sensors installed on the vehicle. When the initial posture represents a non-standard posture, image acquisition prompt information is provided to prompt the user to further verify the posture of the vehicle by taking a photo of the vehicle. After acquiring the captured image, image recognition is performed on the captured vehicle image to obtain an object recognition result including the location information of the vehicle and the reference object, and the posture information of the vehicle is determined based on the object recognition result. In this way, the accuracy of the vehicle posture verification can be further improved, avoiding the situation where the user adjusts the vehicle posture multiple times and it is still judged as a non-standard posture, thereby improving the user experience. At the same time, this embodiment can achieve accurate posture verification without adding sensors to the vehicle, and can also adapt to the vehicles that have been deployed, reducing the dependence of vehicle posture judgment on hardware such as sensors, and reducing maintenance costs.

[0142] exist Figure 7 and Figure 8In the embodiment, data is transmitted between the vehicle and the user terminal through the server. It should be understood that the method of this embodiment can also be implemented in application scenarios where the user terminal and the vehicle can communicate directly, for example, the user terminal directly sends a request to confirm the return of the vehicle to the target vehicle, and the target vehicle directly sends image acquisition prompt information to the user terminal.

[0143] Further, Figure 7 and Figure 8 The embodiment is merely an exemplary three-end interaction method. Since the central control, user terminal and server of the target vehicle are all equipped with processors and have the ability to process related steps, this embodiment does not limit the execution end of each step and can be configured based on specific application scenarios.

[0144] Fig. 9 Schematic diagram of a vehicle posture detection device according to an embodiment of the present invention. Fig. 9 As shown, the vehicle posture detection device 9 of this embodiment includes an image acquisition unit 91, an image recognition unit 92, a posture detection unit 93 and a parking management unit 94.

[0145] The image acquisition unit 91 is configured to acquire a target image, wherein the target image includes a target vehicle in a parking verification state. The image recognition unit 92 is configured to perform image recognition on the target image and acquire an object recognition result, wherein the object recognition result at least includes position information of the target vehicle and a reference object. The posture detection unit 93 is configured to determine the posture information of the target vehicle according to the object recognition result. The parking management unit 94 is configured to determine that the target vehicle is parked in a standardized manner in response to the posture information representing a standard posture.

[0146] In an optional implementation, the image recognition unit 92 is further configured to input the target image into a pre-trained object recognition model for image processing to obtain the object recognition result.

[0147] In an optional implementation, the object recognition model is an image segmentation model, and the image segmentation model includes an image coding layer and a convolution layer.

[0148] The image recognition unit 92 is further configured to perform image data processing on the target image through the image coding layer to obtain image feature information, and process the image feature information through the convolution layer to obtain the object recognition result.

[0149] In an optional implementation, the reference object is a combination of one or more of the following: a road edge sign, a parking area edge line, and at least one parked vehicle other than the target vehicle.

[0150] In an optional implementation, the posture detection unit 93 is further configured to input the target image and the object recognition result into a pre-trained posture recognition model for processing to obtain the posture information of the target vehicle.

[0151] In an optional implementation, the vehicle posture detection device 9 further includes an information generating unit configured to generate image acquisition prompt information in response to the driving data corresponding to the target vehicle characterizing that the initial posture of the target vehicle is a non-standard posture.

[0152] In an optional implementation, the vehicle posture detection device 9 is deployed in the user terminal, and the image acquisition prompt information includes prompt content and image acquisition controls. The image acquisition unit 91 is further configured to call the camera in the user terminal in response to the image acquisition control being triggered, and acquire the target image in response to the shooting control being triggered.

[0153] In an optional implementation, the vehicle posture detection device 9 is deployed on a server or a vehicle, and the image acquisition unit 91 is further configured to send the image acquisition prompt information to the user terminal and receive the target image acquired by the user terminal.

[0154] This embodiment obtains a target image of a target vehicle in a parking verification state, performs image recognition on the target image, obtains an object recognition result including the position information of the target vehicle and a reference object, determines the posture information of the target vehicle according to the object recognition result, and determines the standard parking of the target vehicle in response to the standard posture represented by the posture information. Thus, this embodiment can improve the judgment accuracy of the vehicle posture by identifying and analyzing the features of the image, and reduce the dependence of the vehicle posture judgment on hardware such as sensors, thereby reducing the maintenance cost.

[0155] Fig.10 Schematic diagram of an electronic device according to an embodiment of the present invention. The electronic device according to this embodiment may be the user terminal, or a server, or a central control device of a vehicle. Fig.10 As shown, Fig.10The electronic device shown is a general address query device, which includes a general computer hardware structure, which at least includes a processor 101 and a memory 102. The processor 101 and the memory 102 are connected via a bus 103. The memory 102 is suitable for storing instructions or programs executable by the processor 101. The processor 101 can be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 101 executes the instructions stored in the memory 102, thereby executing the method flow of the embodiment of the present invention as described above to realize the processing of data and the control of other devices. The bus 103 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 104 and the display device and the input / output (I / O) device 105. The input / output (I / O) device 105 can be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, the input / output device 105 is connected to the system through an input / output (I / O) controller 106.

[0156] Furthermore, if the electronic device is a user terminal, the electronic device further includes a camera to capture a target image.

[0157] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices (equipment) or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may adopt a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0158] The present application is described with reference to flowcharts of methods, apparatuses (devices) and computer program products according to embodiments of the present application. It should be understood that each process in the flowchart can be implemented by computer program instructions.

[0159] These computer program instructions may be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device that implements the process Figure 1 A function specified in a process or multiple processes.

[0160] These computer program instructions may also be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce the instructions for implementing the process Figure 1 A device that specifies functions in a process or multiple processes.

[0161] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program, wherein the computer-readable program is used for a computer to execute part or all of the above method embodiments.

[0162] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by specifying relevant hardware through a program, and the program is stored in a storage medium, including several instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0163] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A vehicle posture detection method, characterized in that: The method comprises: Acquire a target image, wherein the target image includes a target vehicle in a parking verification state; Performing image recognition on the target image to obtain an object recognition result, wherein the object recognition result at least includes position information of the target vehicle and a reference object; Determining the posture information of the target vehicle according to the object recognition result; In response to the posture information representing a standard posture, it is determined that the target vehicle is parked in a standard manner.

2. The method according to claim 1, characterized in that The performing image recognition on the target image to obtain an object recognition result comprises: The target image is input into a pre-trained object recognition model for image processing to obtain the object recognition result.

3. The method according to claim 2, characterized in that The object recognition model is an image segmentation model, and the image segmentation model includes an image coding layer and a convolution layer; The inputting the target image into a pre-trained object recognition model for image processing to obtain the object recognition result comprises: Performing image data processing on the target image through the image coding layer to obtain image feature information; The image feature information is processed through the convolution layer to obtain the object recognition result.

4. The method according to claim 1, characterized in that: The reference object is a combination of one or more of the following: a road edge sign, a parking area edge line, and at least one parked vehicle other than the target vehicle.

5. The method according to claim 1, characterized in that Determining the posture information of the target vehicle according to the object recognition result includes: The target image and the object recognition result are input into a pre-trained posture recognition model for processing to obtain the posture information of the target vehicle.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: In response to the driving data corresponding to the target vehicle indicating that the initial posture of the target vehicle is a non-standard posture, image acquisition prompt information is generated.

7. The method according to claim 6, characterized in that The method is applied to a user terminal, and the image acquisition prompt information includes prompt content and image acquisition controls; The acquiring of the target image comprises: In response to the image acquisition control being triggered, calling a camera in the user terminal; In response to the shooting control being triggered, the target image is acquired.

8. The method according to claim 6, characterized in that The method is applied to a server or a vehicle, and acquiring a target image comprises: Sending the image acquisition prompt information to the user terminal; Receive the target image captured by the user terminal.

9. A vehicle posture detection device, characterized in that: The device comprises: An image acquisition unit is configured to acquire a target image, wherein the target image includes a target vehicle in a parking verification state; an image recognition unit, configured to perform image recognition on the target image and obtain an object recognition result, wherein the object recognition result at least includes position information of the target vehicle and a reference object; a posture detection unit, configured to determine the posture information of the target vehicle according to the object recognition result; The parking management unit is configured to determine that the target vehicle is parked in a standardized manner in response to the posture information representing a standard posture.

10. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product, characterized in that When the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 8.

Images