Vehicle position determination method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application provide a vehicle position determination method and device, electronic equipment and storage medium, comprising: determining a target floor corresponding to a to-be-parked position of a vehicle according to information collected by an inertial measurement unit and a tunnel video stream collected by a camera; when detecting that a user has engaged a reverse gear, controlling the camera to record a video stream until the vehicle stops to obtain a reverse video stream; obtaining a target surround-view stereoscopic image containing the target floor according to the reverse video stream and the target floor; and sending the target surround-view stereoscopic image to the user, so that the user determines the parking position of the vehicle according to the target surround-view stereoscopic image. The accuracy of the confirmed target floor can be improved, and the user can quickly determine the position of the vehicle according to the target surround-view stereoscopic image, thereby improving the efficiency of finding the vehicle.

Description

Technical Field

[0001] This application relates to the field of vehicle location technology, and in particular to a method, apparatus, electronic device and storage medium for determining vehicle location. Background Technology

[0002] With the continuous increase in car ownership, public parking lots are becoming increasingly large and complex, making it more and more difficult for car owners to find their cars after parking.

[0003] After driving into the parking lot, users can use the in-vehicle camera to record video of the surrounding environment and perform OCR recognition on the video to obtain information about the floor and parking space where the vehicle stopped. This information, including the floor and parking space details, will then be sent to the user.

[0004] However, if users are unfamiliar with the parking lot environment, determining the vehicle's location solely based on fields containing floor and parking space information is extremely inefficient. Summary of the Invention

[0005] This application provides a vehicle location determination method, apparatus, electronic device, and storage medium to address the problem of inefficient vehicle location determination in parking lots.

[0006] In a first aspect, embodiments of this application provide a method for determining the location of a vehicle, including:

[0007] The target floor corresponding to the vehicle's parking position is determined based on the information collected by the inertial measurement unit and the video stream of the tunnel collected by the camera.

[0008] When the system detects that the user has engaged reverse gear, it controls the camera to record a video stream until the vehicle stops, thus obtaining a reversing video stream.

[0009] Based on the reversing video stream and the target floor, a surround-view stereoscopic image of the target floor is obtained;

[0010] The target surround-view stereo image is sent to the user so that the user can determine the parking position of the vehicle based on the target surround-view stereo image.

[0011] Optionally, the target floor corresponding to the vehicle's parking position can be determined based on information collected by the inertial measurement unit and video stream from the tunnel captured by the camera, including:

[0012] Acquire a video stream of the passageway captured by a camera, the video stream of the passageway containing text displayed in the passageway to indicate the floors;

[0013] Text recognition is performed on the images in the tunnel video stream to determine the first floor information when the vehicle leaves the tunnel;

[0014] Based on the information collected by the inertial measurement unit, the second floor information when the vehicle leaves the passageway is determined;

[0015] Based on the first floor information and the second floor information, determine the target floor corresponding to the vehicle's parking location.

[0016] Optionally, text recognition is performed on the images in the tunnel video stream to determine the first floor information when the vehicle leaves the tunnel, including:

[0017] Keyframes are extracted from the tunnel video stream to obtain multiple keyframe images, which are images containing text.

[0018] Text recognition is performed sequentially on the multiple keyframe images in order of shooting time from back to front. If a preset number of consecutive keyframe images contain the same text, the same text is determined to be text used to indicate information about the first floor, and the recognition operation is stopped.

[0019] Optionally, based on the information collected by the inertial measurement unit, the second floor information when the vehicle leaves the passageway is determined, including:

[0020] Based on the tunnel video stream, determine the first pose change of the vehicle when leaving the tunnel relative to when entering the tunnel;

[0021] Based on the information collected by the inertial measurement unit, the second pose change of the vehicle when leaving the tunnel relative to when entering the tunnel is determined;

[0022] The first pose change and the second pose change are fused to determine the second floor information when the vehicle leaves the passageway.

[0023] Optionally, based on the tunnel video stream, determine the first pose change of the vehicle when leaving the tunnel relative to when entering the tunnel, including:

[0024] The sliding window method was used to extract key frames from the tunnel video stream, resulting in multiple pose key frame images;

[0025] The iterative nearest point method is used to solve the multiple pose keyframes to determine the change in the first pose of the vehicle when it leaves the passageway relative to when it enters the passageway.

[0026] Optionally, based on the reversing video stream and the target floor, a target surround-view stereoscopic image including the target floor is obtained, including:

[0027] Based on the reversing video stream, a surround-view stereo image containing parking space information is obtained;

[0028] The target floor is added to the surround-view stereo image to obtain a target surround-view stereo image containing the target floor.

[0029] Optionally, based on the reversing video stream, a surround-view stereoscopic image containing parking space information is obtained, including:

[0030] Keyframes are selected from the reversing video stream to obtain parking space keyframes with parking space information;

[0031] The parking space keyframes are stitched to the bottom of the last frame of the reversing video stream to obtain a surround-view stereoscopic image containing parking space information.

[0032] Secondly, embodiments of this application provide a vehicle location determination device, comprising:

[0033] The determination module is used to determine the target floor corresponding to the vehicle's parking position based on the information collected by the inertial measurement unit and the tunnel video stream collected by the camera;

[0034] The control module is used to control the camera to record a video stream until the vehicle stops when it detects that the user has engaged reverse gear, thus obtaining a reversing video stream;

[0035] The module obtains a target surround-view stereoscopic image containing the target floor based on the reversing video stream and the target floor.

[0036] The sending module sends the target surround-view stereoscopic image to the user so that the user can determine the parking position of the vehicle based on the target surround-view stereoscopic image.

[0037] Thirdly, embodiments of this application provide an electronic device, including:

[0038] At least one processor; and a memory communicatively connected to said at least one processor;

[0039] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, cause the electronic device to perform the method described in any of the above aspects.

[0040] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method described in any of the above aspects.

[0041] The vehicle location determination method, apparatus, electronic device, and storage medium provided in this application include: determining the target floor corresponding to the vehicle's parking position based on information collected by an inertial measurement unit and a tunnel video stream collected by a camera; controlling the camera to record a video stream until the vehicle stops when the user is detected to be in reverse gear, thus obtaining a reversing video stream; obtaining a target surround-view stereoscopic image containing the target floor based on the reversing video stream and the target floor; and sending the target surround-view stereoscopic image to the user so that the user can determine the vehicle's parking position based on the target surround-view stereoscopic image. This can improve the accuracy of the identified target floor, and the user can quickly determine the vehicle's position based on the target surround-view stereoscopic image, thus improving vehicle location efficiency. Attached Figure Description

[0042] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0043] Figure 1 An application scenario diagram provided for an embodiment of this application;

[0044] Figure 2 This is a schematic flowchart of a vehicle location determination method provided in an embodiment of this application;

[0045] Figure 3 A schematic diagram illustrating the process of a vehicle entering a target floor from outside a parking lot via a passageway, as provided in an embodiment of this application;

[0046] Figure 4 A flowchart illustrating a user vehicle-finding method provided in an embodiment of this application;

[0047] Figure 5 Another method for determining vehicle location provided in this application embodiment;

[0048] Figure 6 This is a schematic diagram of the structure of a vehicle position determination device provided in an embodiment of this application;

[0049] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0050] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0051] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0052] As parking lots become increasingly larger, with multiple floors, vast spaces, and difficult-to-distinguish directions, they resemble underground mazes, making it increasingly difficult for car owners to find their vehicles after parking.

[0053] In some technologies, after a user drives their vehicle into a parking lot, the vehicle's camera can record video of the surrounding environment. The video frames can then be OCR-recognized to identify the floor and parking space where the vehicle stopped. The identified information, including the floor and parking space details, is then sent to the user so they can locate their vehicle based on the received information.

[0054] However, the accuracy of the floor and parking space information obtained by using OCR recognition is not high, and the efficiency of determining the vehicle's location based solely on the received field information is very low when the user is not particularly familiar with the parking lot environment.

[0055] In view of this, this application provides a vehicle location determination method. First, based on information collected by an inertial measurement unit and a video stream from a camera capturing a passageway, the target floor corresponding to the vehicle's intended parking position is determined. Then, based on a reversing video stream captured by the camera and the target floor, a target surround-view stereoscopic image is obtained. This target surround-view stereoscopic image is a stereoscopic image containing floor information, parking space information, and the surrounding environment at the time of vehicle parking. The target surround-view stereoscopic image is sent to the user so that the user can determine the vehicle's parking position based on it. This improves the accuracy of the identified target floor, increases vehicle location efficiency, and enhances the user experience.

[0056] Figure 1This is an application scenario diagram provided by an embodiment of this application. It includes an in-vehicle terminal and a user device. The executing entity of this application can be the central control system of the in-vehicle terminal. When the central control system detects that a vehicle is moving in a parking lot aisle, it controls the camera and inertial measurement unit to collect corresponding information, obtaining an aisle video stream based on the information collected by the camera. The target floor information can be determined based on the aisle video stream collected by the camera and the information collected by the inertial measurement unit. When the central control system detects that the user has engaged reverse gear, it controls the camera to record a video stream again until the vehicle stops, obtaining a reversing video stream. Based on the reversing video stream and the target floor, a target surround-view stereoscopic image is obtained. The target surround-view stereoscopic image contains parking space information, floor information, and information about the surrounding environment. The target surround-view stereoscopic image is sent to the user terminal so that the user can determine the parking position of the vehicle based on the target surround-view stereoscopic image.

[0057] The vehicle location determination method provided in this application utilizes information collected by an inertial measurement unit and video streams from a camera to jointly determine the target floor corresponding to the vehicle's parking position, thereby improving the accuracy of the identified target floor. Based on the reversing video stream recorded by the camera and the target floor, a target surround-view stereoscopic image is obtained. This target surround-view stereoscopic image includes floor information, parking space information, and the surrounding environment when the vehicle is parked. The target surround-view stereoscopic image is sent to the user so that the user can determine the vehicle's parking position based on it. In addition to providing floor and parking space information, the target surround-view stereoscopic image further recreates the user's parking scenario, providing an immersive experience and enabling the user to quickly determine the vehicle's location based on their surroundings, thus improving vehicle search efficiency.

[0058] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0059] Figure 2 This is a schematic flowchart illustrating a vehicle location determination method provided in an embodiment of this application. Figure 2 As shown in the embodiments of this application, a vehicle location determination method may include:

[0060] Step 201: Determine the target floor corresponding to the vehicle's parking position based on the information collected by the inertial measurement unit and the tunnel video stream collected by the camera.

[0061] The tunnel video stream is a video stream captured by onboard cameras as vehicles move through the parking lot tunnels. The inertial measurement unit (IMU) is a crucial component of the inertial guidance system (INS), capable of measuring an object's three-axis attitude angles (or angular rates) and acceleration.

[0062] The specific steps by which the central control system detects vehicle movement in the parking garage aisle are as follows: When a vehicle enters the aisle from outside the parking garage, the three-axis acceleration measured by the gyroscope in the INS will show a significant change. When this change exceeds a preset threshold C, it is considered that the vehicle has begun to enter the parking garage aisle, and the INS outputs the signal ImuIPT at this time. When the vehicle moves from the aisle to the target floor corresponding to the parking position, the three-axis acceleration measured by the gyroscope in the INS will also show a significant change. When this change exceeds a preset threshold C, it is considered that the vehicle has left the aisle and entered the target floor, and the INS outputs the signal ImuOPT at this time. Between the signals ImuIPT and ImuOPT, the vehicle is considered to be moving in the parking garage aisle.

[0063] Figure 3 This is a schematic diagram illustrating the process of a vehicle entering a target floor from outside a parking lot via a passageway, as provided in an embodiment of this application. The entire process is divided into three stages: in the first stage, the vehicle moves on flat ground outside the parking lot; in the second stage, the vehicle moves in the passageway; and in the third stage, the vehicle moves on flat ground on the target floor. When the vehicle leaves the flat ground outside the parking lot and enters the passageway, the three-axis acceleration measured by the gyroscope in the INS will change significantly, and this change will exceed a preset threshold C. The INS outputs the signal ImuIPT at this time. When the vehicle leaves the passageway and enters the target floor, the three-axis acceleration measured by the gyroscope in the INS will also change significantly, and this change will exceed the preset threshold C. The INS outputs the signal ImuOPT at this time.

[0064] Specifically, based on the information collected by the inertial measurement unit (IMU) during the vehicle's movement in the parking lot aisle and the video stream of the aisle captured by the camera, the target floor corresponding to the vehicle's intended parking position is determined. The information collected by the IMU includes the vehicle's three-axis attitude angles (or angular velocities) and acceleration at each moment.

[0065] Optionally, the target floor corresponding to the vehicle's parking position can be determined based on information collected by the inertial measurement unit and video stream from the tunnel captured by the camera, including:

[0066] Acquire a video stream of the passageway captured by a camera, the video stream of the passageway containing text displayed in the passageway to indicate the floors;

[0067] Text recognition is performed on the images in the tunnel video stream to determine the first floor information when the vehicle leaves the tunnel;

[0068] Based on the information collected by the inertial measurement unit, the second floor information when the vehicle leaves the passageway is determined;

[0069] Based on the first floor information and the second floor information, determine the target floor corresponding to the vehicle's parking location.

[0070] Specifically, the central control system first acquires the video stream of the passageway captured by the camera. The passageway is equipped with text indicating the floors, so the video stream captured by the camera will contain the corresponding text, such as B1, B2, and B3, representing basement level 1, basement level 2, and basement level 3, respectively. By performing text recognition on the images in the passageway video stream, the floor information when the vehicle leaves the passageway and enters the target floor can be obtained. This floor information is called the first floor information. Based on the information collected by the inertial measurement unit, the INS can determine the vehicle's pose changes during its movement in the passageway, and thus determine the vehicle's vertical displacement. Based on the vehicle's vertical displacement during its movement in the passageway, the floor information when the vehicle leaves the passageway and enters the target floor can also be determined. This floor information is called the second floor information.

[0071] Assuming an underground parking garage has three underground levels, in one possible implementation, the floor information of the vehicle when it leaves the passageway and enters the target floor is determined based on the vertical displacement of the vehicle during its movement in the passageway, including:

[0072] When the vertical displacement is less than the first preset displacement, the floor information when the vehicle leaves the passageway and enters the target floor is determined to be the basement level 1.

[0073] When the vertical displacement is greater than the first preset displacement and less than the second preset displacement, the floor information when the vehicle leaves the passageway and enters the target floor is determined to be the second basement level.

[0074] When the vertical displacement is greater than the second preset displacement, the floor information when the vehicle leaves the passageway and enters the target floor is determined to be the 3rd basement level.

[0075] For example, suppose an underground parking lot has three underground levels. The first preset displacement and the second preset displacement are set to 2.5m and 5m respectively. When the vertical displacement of the vehicle during its movement in the passageway is calculated to be 4m, since 2.5m < 4m < 5m, the floor information when the vehicle leaves the passageway and enters the target floor is determined to be the second underground level. When the vertical displacement of the vehicle during its movement in the passageway is calculated to be 7m, since 7m > 5m, the floor information when the vehicle leaves the passageway and enters the target floor is determined to be the third underground level.

[0076] When the underground parking garage is not three stories high, the above method can also be used to determine the floor information when the vehicle leaves the passageway and enters the target floor based on the vertical displacement of the vehicle during its movement in the passageway. This will not be elaborated further here.

[0077] After determining the first and second floor information, it is checked whether the first and second floor information are the same. If they are the same, it is determined that the target floor corresponding to the vehicle's parking position is the same as the first or second floor information. If they are different, an error is reported, and relevant information is sent to the backend server and an alarm is triggered so that the backend staff can carry out maintenance as soon as possible.

[0078] In this way, the first floor information is determined by text recognition of the video stream in the passageway, and the vertical displacement is determined based on the information collected by the inertial measurement unit, thereby determining the second floor information. When the first floor information and the second floor information are the same, the target floor corresponding to the vehicle's parking position is determined. Compared with the prior art of determining the target floor by text recognition of the video stream, the method of this application can improve the accuracy of determining the target floor.

[0079] Optionally, text recognition is performed on the images in the tunnel video stream to determine the first floor information when the vehicle leaves the tunnel, including:

[0080] Keyframes are extracted from the tunnel video stream to obtain multiple keyframe images, which are images containing text.

[0081] Text recognition is performed sequentially on the multiple keyframe images in order of shooting time from back to front. If a preset number of consecutive keyframe images contain the same text, the same text is determined to be text used to indicate information about the first floor, and the recognition operation is stopped.

[0082] Specifically, the central control system extracts keyframes from the tunnel video stream to obtain multiple keyframe images. The extracted keyframe images all contain text. Different methods can be used to extract keyframes from the tunnel video stream, and this application does not limit this method.

[0083] In one alternative implementation, a keyframe extraction model can be trained using deep learning. The input to the keyframe extraction model is a video stream, and the output is the images of all video frames in the video stream that contain text, i.e., keyframe images.

[0084] After obtaining multiple keyframe images, the multiple keyframe images are sorted in order from the last captured time to the first captured time to obtain a keyframe image sequence. Text recognition is then performed on the keyframe images in the keyframe sequence in turn to obtain the corresponding text. The text contains floor information, and the obtained text can be B1, B2, B3, etc. If a preset number of consecutive keyframe images contain the same text, the identified identical text is used as the text indicating the first floor information, and the recognition of subsequent keyframe images is stopped.

[0085] First, keyframes are selected from the video stream of the passageway. Then, text recognition is performed on the selected keyframe images. Finally, the text used to indicate the information of the first floor is determined based on the multiple recognized texts. In this way, since the selected keyframe images all contain text, text recognition is only performed on the keyframe images, which improves the efficiency of text recognition without losing useful information.

[0086] Optionally, based on the information collected by the inertial measurement unit, the second floor information when the vehicle leaves the passageway is determined, including:

[0087] Based on the tunnel video stream, determine the first pose change of the vehicle when leaving the tunnel relative to when entering the tunnel;

[0088] Based on the information collected by the inertial measurement unit, the second pose change of the vehicle when leaving the tunnel relative to when entering the tunnel is determined;

[0089] The first pose change and the second pose change are fused to determine the second floor information when the vehicle leaves the passageway.

[0090] Specifically, based on the changes in objects in any frame of the tunnel video stream relative to the previous frame, the vehicle's pose change during the time interval between capturing these two frames can be determined. Therefore, based on the tunnel video stream, the vehicle's pose change when leaving the tunnel relative to when entering the tunnel can be determined; this change is called the first pose change. The INS system calculates the vehicle's pose change when leaving the tunnel relative to when entering the tunnel by integrating the information collected by the inertial measurement unit; this pose change is called the second pose change. By tightly coupling the first and second pose changes, the vertical displacement of the vehicle during its journey from entering to leaving the tunnel can be obtained. Based on this vertical displacement, the second floor information when the vehicle leaves the tunnel can be determined.

[0091] When determining the first pose change based on the tunnel video stream, the advantage is that it does not produce drift and can directly measure rotation and translation. The disadvantage is that it is greatly affected by the image and features are easily lost during rapid motion. When determining the second pose change based on the information collected by the inertial measurement unit, the advantage is that the response speed is fast and the measured acceleration is accurate. The disadvantage is that there is cumulative error, and the accuracy of the result decreases the longer the time. Therefore, fusing the first pose change and the second pose change can compensate for the shortcomings of the two methods and improve the accuracy of the confirmed second floor information.

[0092] Optionally, based on the tunnel video stream, determine the first pose change of the vehicle when leaving the tunnel relative to when entering the tunnel, including:

[0093] The sliding window method was used to extract key frames from the tunnel video stream, resulting in multiple pose key frame images;

[0094] The iterative nearest point method is used to solve the multiple pose keyframes to determine the change in the first pose of the vehicle when it leaves the passageway relative to when it enters the passageway.

[0095] Specifically, the central control system uses the sliding window method to extract keyframes from the tunnel video stream, obtaining multiple pose keyframe images. The ORB (Oriented FAST and Rotated BRIEF) algorithm is used to extract angular features from the pose keyframe images. The Iterative Closest Point (ICP) method is used to solve the multiple pose keyframes to determine the first pose change of the vehicle when leaving the tunnel relative to when entering the tunnel.

[0096] In this way, by extracting keyframes from the tunnel video stream using the sliding window method and solving the multiple pose keyframes using the iterative nearest point method to determine the first pose change, the efficiency of determining the first pose change can be improved.

[0097] Step 202: When the user is detected to have engaged reverse gear, control the camera to record a video stream until the vehicle stops, thus obtaining a reversing video stream.

[0098] Specifically, when the central control system detects that the user has engaged reverse gear, it controls the onboard cameras to record video until the vehicle comes to a stop, resulting in a reversing video stream. Multiple onboard cameras can be used; when the central control system detects that the user has engaged reverse gear, it controls multiple onboard cameras to record video until the vehicle stops, thus obtaining multiple reversing video streams.

[0099] Step 203: Based on the reversing video stream and the target floor, obtain a target surround-view stereoscopic image containing the target floor.

[0100] The target surround-view stereo image includes environmental information around the vehicle when it is stationary and the target floor.

[0101] Specifically, based on the reversing video stream and the identified target floor, a surround-view stereoscopic image of the target floor can be obtained.

[0102] Optionally, based on the reversing video stream and the target floor, a target surround-view stereoscopic image including the target floor is obtained, including:

[0103] Based on the reversing video stream, a surround-view stereo image containing parking space information is obtained;

[0104] The target floor is added to the surround-view stereo image to obtain a target surround-view stereo image containing the target floor.

[0105] Specifically, based on the reversing video stream, a surround-view stereo image is obtained. This image includes environmental information and parking space information around the vehicle when it is stationary. By printing the target floor onto the surround-view stereo image, a target surround-view stereo image containing the target floor can be obtained.

[0106] Optionally, based on the reversing video stream, a surround-view stereoscopic image containing parking space information is obtained, including:

[0107] Keyframes are selected from the reversing video stream to obtain parking space keyframes with parking space information;

[0108] The parking space keyframes are stitched to the bottom of the last frame of the reversing video stream to obtain a surround-view stereoscopic image containing parking space information.

[0109] Specifically, a parking space keyframe selection model can be trained using deep learning. This model can then be used to select keyframes from the reversing video stream. The reversing video stream can be input into the parking space keyframe selection model, which will output parking space keyframes containing parking space information.

[0110] The central control system controls multiple cameras on the vehicle to record multiple reversing video streams. For each reversing video stream, the last frame of that reversing video stream is taken to obtain multiple last frame images. These multiple last frame images are then stitched together in a ring to obtain a stereoscopic image of the surrounding environment of the vehicle's parking position from the camera angle. Finally, the parking space keyframe is stitched onto the bottom of the stereoscopic image to obtain a surround stereoscopic image containing parking space information.

[0111] In this way, the target surround-view stereoscopic image not only provides users with floor and parking space information, but also further restores the user's parking scenario, giving users an immersive experience and enabling them to quickly determine the vehicle's location based on their surroundings, thus improving the efficiency of finding their car.

[0112] Step 204: Send the target surround-view stereo image to the user so that the user can determine the parking position of the vehicle based on the target surround-view stereo image.

[0113] Specifically, the central control system sends the target surround-view stereo image to the user. After receiving the target surround-view stereo image, the user can open it and swipe up and down or left and right to obtain the environmental information around the vehicle from the camera angle when the vehicle is parked. The bottom also stitches together the parking space keyframes, giving the user an immersive experience. The parking position is determined based on the floor information, parking space information and the environmental information around the vehicle's parking location in the target surround-view stereo image.

[0114] Figure 4 This is a flowchart illustrating a user vehicle-finding method provided in an embodiment of this application, as shown below. Figure 4 As shown, the user begins the search for their car, takes out their phone to view the target surround-view 3D image, and determines the car's location based on the floor information, parking space information, and environmental information around the vehicle's parking location in the target surround-view 3D image, thus ending the search.

[0115] In summary, this application provides a vehicle location determination method, including: determining the target floor corresponding to the vehicle's parking position based on information collected by an inertial measurement unit and a tunnel video stream collected by a camera, which can improve the accuracy of the identified target floor; when the user is detected to have engaged reverse gear, controlling the camera to record a video stream until the vehicle stops, obtaining a reversing video stream; obtaining a target surround-view stereoscopic image containing the target floor based on the reversing video stream and the target floor; and sending the target surround-view stereoscopic image to the user so that the user can determine the vehicle's parking position based on the target surround-view stereoscopic image, enabling the user to quickly determine the vehicle's parking position and improving vehicle search efficiency.

[0116] Figure 5Another vehicle position determination method provided in this application embodiment involves a central control system controlling a camera and an inertial measurement unit to collect relevant information when the system detects a vehicle moving in a parking lot passageway. The system obtains a passageway video stream based on the information collected by the camera. The central control system extracts keyframes from the passageway video stream using a sliding window method. The selected keyframe images have two uses: first, to determine the first floor information when the vehicle leaves the passageway; specifically, OCR recognition is performed on the selected keyframes to obtain multiple texts, and the text indicating the first floor information is selected from these texts. Second, to determine the first pose change of the vehicle when leaving the passageway relative to when it entered the passageway; specifically, the ORB algorithm is used to extract angular features from the selected keyframe images, and ICP is performed between adjacent keyframes to obtain the first pose change. Based on the information collected by the inertial measurement unit, the second pose change of the vehicle when leaving the passageway relative to when entering the passageway can be determined. The first pose change and the second pose change are tightly coupled to determine the second floor information when the vehicle leaves the passageway. It is then determined whether the first floor information and the second floor information are the same. If they are the same, the target floor information corresponding to the vehicle's parking position is determined to be the same as the first floor information or the second floor information. When the central control system detects that the user has engaged reverse gear, it controls multiple onboard cameras to start recording video streams until the vehicle stops, obtaining multiple reversing video streams. Keyframes are selected from the multiple reversing video streams to obtain parking space keyframes containing parking space information. The last frame of the multiple reversing video streams is stitched together to obtain a stereoscopic image of the environment around the vehicle's parking position. The parking space keyframes are stitched to the bottom of the stereoscopic image to obtain a surround-view stereoscopic image containing parking space information. The determined target floor information is then printed into the surround-view stereoscopic image to obtain a target surround-view stereoscopic image. The target surround-view stereoscopic image is sent to the user so that the user can determine the vehicle's parking position based on the target surround-view stereoscopic image.

[0117] Corresponding to the above-described vehicle location determination method, this application also provides a vehicle location determination device. Figure 6 This is a schematic diagram of a vehicle position determination device provided in an embodiment of this application. Figure 6 As shown, the device includes:

[0118] The determination module 601 is used to determine the target floor corresponding to the vehicle's parking position based on the information collected by the inertial measurement unit and the tunnel video stream collected by the camera.

[0119] Control module 602 is used to control the camera to record a video stream until the vehicle stops when it detects that the user has engaged reverse gear, thus obtaining a reversing video stream;

[0120] Module 603 obtains a target surround-view stereoscopic image containing the target floor based on the reversing video stream and the target floor.

[0121] The sending module 604 sends the target surround-view stereo image to the user so that the user can determine the parking position of the vehicle based on the target surround-view stereo image.

[0122] Optionally, module 601 is determined, specifically for:

[0123] Acquire a video stream of the passageway captured by a camera, the video stream of the passageway containing text displayed in the passageway to indicate the floors;

[0124] Text recognition is performed on the images in the tunnel video stream to determine the first floor information when the vehicle leaves the tunnel;

[0125] Based on the information collected by the inertial measurement unit, the second floor information when the vehicle leaves the passageway is determined;

[0126] Based on the first floor information and the second floor information, determine the target floor corresponding to the vehicle's parking location.

[0127] Optionally, when determining the first floor information of the vehicle when it leaves the tunnel, the determining module 601 performs text recognition on the images in the tunnel video stream. Specifically, it is used for:

[0128] Keyframes are extracted from the tunnel video stream to obtain multiple keyframe images, which are images containing text.

[0129] The text recognition is performed on the multiple keyframe images in order of shooting time from back to front. If a preset number of consecutive keyframe images contain the same text, the same text is determined to be text used to indicate the information of the first floor, and the recognition operation is stopped.

[0130] Optionally, the determining module 601 determines the second floor information when the vehicle leaves the passageway based on the information collected by the inertial measurement unit, specifically for:

[0131] Based on the tunnel video stream, determine the first pose change of the vehicle when leaving the tunnel relative to when entering the tunnel;

[0132] Based on the information collected by the inertial measurement unit, the second pose change of the vehicle when leaving the tunnel relative to when entering the tunnel is determined;

[0133] The first pose change and the second pose change are fused to determine the second floor information when the vehicle leaves the passageway.

[0134] Optionally, when determining the first pose change of the vehicle when leaving the tunnel relative to when entering the tunnel, based on the tunnel video stream, the determining module 601 is specifically used for:

[0135] The sliding window method was used to extract key frames from the tunnel video stream, resulting in multiple pose key frame images;

[0136] The iterative nearest point method is used to solve the multiple pose keyframes to determine the change in the first pose of the vehicle when it leaves the passageway relative to when it enters the passageway.

[0137] Optionally, module 603 is obtained, specifically for:

[0138] Based on the reversing video stream, a surround-view stereoscopic image containing parking space information is obtained;

[0139] The target floor is added to the surround-view stereo image to obtain a target surround-view stereo image containing the target floor.

[0140] Optionally, when obtaining the surround-view stereoscopic image containing parking space information based on the reversing video stream, module 603 is specifically used for:

[0141] Keyframes are selected from the reversing video stream to obtain parking space keyframes with parking space information;

[0142] The parking space keyframes are stitched to the bottom of the last frame of the reversing video stream to obtain a surround-view stereoscopic image containing parking space information.

[0143] The specific implementation principle and effects of the vehicle location determination device provided in this application can be found in the foregoing embodiments, and will not be repeated here.

[0144] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device in this embodiment may include:

[0145] At least one processor 701; and

[0146] Memory 702 communicatively connected to the at least one processor;

[0147] The memory 702 stores instructions that can be executed by the at least one processor 701 to cause the electronic device to perform the method as described in any of the above embodiments.

[0148] Alternatively, the memory 702 can be either standalone or integrated with the processor 701.

[0149] The implementation principle and technical effects of the electronic device provided in this embodiment can be found in the foregoing embodiments, and will not be repeated here.

[0150] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method described in any of the foregoing embodiments.

[0151] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the methods described in any of the foregoing embodiments.

[0152] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0153] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application.

[0154] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor. The memory may include high-speed RAM, and may also include non-volatile memory (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk, or optical disc, etc.

[0155] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0156] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. Both the processor and the storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic device or host device.

[0157] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0158] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0159] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0160] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for determining the location of a vehicle, characterized in that, include: The target floor corresponding to the vehicle's parking position is determined based on the information collected by the inertial measurement unit and the video stream of the tunnel collected by the camera. When the system detects that the user has engaged reverse gear, it controls the camera to record a video stream until the vehicle stops, thus obtaining a reversing video stream. Based on the reversing video stream and the target floor, a target surround stereoscopic image containing the target floor is obtained. The target surround stereoscopic image is a video stream that includes the target floor, parking space information, and stereoscopic image of the surrounding environment. The target surround-view stereo image is sent to the user so that the user can determine the parking position of the vehicle based on the target surround-view stereo image; The process of determining the target floor corresponding to the vehicle's parking position based on information collected by the inertial measurement unit and video stream from the tunnel captured by the camera includes: Acquire a video stream of the passageway captured by a camera, the video stream of the passageway containing text displayed in the passageway to indicate the floors; Text recognition is performed on the images in the tunnel video stream to determine the first floor information when the vehicle leaves the tunnel; Based on the information collected by the inertial measurement unit, the second floor information when the vehicle leaves the passageway is determined; If the first floor information and the second floor information are the same, then the first floor information or the second floor information is determined as the target floor corresponding to the vehicle's parking location. The step of determining the second floor information when the vehicle leaves the passageway based on the information collected by the inertial measurement unit includes: Based on the tunnel video stream, determine the first pose change of the vehicle when leaving the tunnel relative to when entering the tunnel; Based on the information collected by the inertial measurement unit, the second pose change of the vehicle when leaving the tunnel relative to when entering the tunnel is determined; The first pose change and the second pose change are fused to determine the second floor information when the vehicle leaves the passageway.

2. The method according to claim 1, characterized in that, Text recognition is performed on the images in the tunnel video stream to determine the first floor information when the vehicle leaves the tunnel, including: Keyframes are extracted from the tunnel video stream to obtain multiple keyframe images, which are images containing text. The text recognition is performed on the multiple keyframe images in order of shooting time from back to front. If a preset number of consecutive keyframe images contain the same text, the same text is determined to be text used to indicate the information of the first floor, and the recognition operation is stopped.

3. The method according to claim 1, characterized in that, Based on the tunnel video stream, determine the vehicle's first pose change when leaving the tunnel relative to when entering the tunnel, including: The sliding window method was used to extract key frames from the tunnel video stream, resulting in multiple pose key frame images; The iterative nearest point method is used to solve the multiple pose keyframes to determine the change in the first pose of the vehicle when it leaves the passageway relative to when it enters the passageway.

4. The method according to any one of claims 1-3, characterized in that, Based on the reversing video stream and the target floor, a target surround-view stereoscopic image including the target floor is obtained, including: Based on the reversing video stream, a surround-view stereoscopic image containing parking space information is obtained; The target floor is added to the surround-view stereo image to obtain a target surround-view stereo image containing the target floor.

5. The method according to claim 4, characterized in that, Based on the reversing video stream, a surround-view stereoscopic image containing parking space information is obtained, including: Keyframes are selected from the reversing video stream to obtain parking space keyframes with parking space information; The parking space keyframes are stitched to the bottom of the last frame of the reversing video stream to obtain a surround-view stereoscopic image containing parking space information.

6. A vehicle position determination device, characterized in that, include: The determination module is used to determine the target floor corresponding to the vehicle's parking position based on the information collected by the inertial measurement unit and the tunnel video stream collected by the camera; The control module is used to control the camera to record a video stream until the vehicle stops when it detects that the user has engaged reverse gear, thus obtaining a reversing video stream; The module obtains a target surround-view stereoscopic image containing the target floor based on the reversing video stream and the target floor. The target surround-view stereoscopic image is a video stream that includes the target floor, parking space information, and stereoscopic images of the surrounding environment. The sending module sends the target surround-view stereo image to the user so that the user can determine the parking position of the vehicle based on the target surround-view stereo image; The determining module is specifically used for: Acquire a video stream of the passageway captured by a camera, the video stream of the passageway containing text displayed in the passageway to indicate the floors; Text recognition is performed on the images in the tunnel video stream to determine the first floor information when the vehicle leaves the tunnel; Based on the information collected by the inertial measurement unit, the second floor information when the vehicle leaves the passageway is determined; If the first floor information and the second floor information are the same, then the first floor information or the second floor information is determined as the target floor corresponding to the vehicle's parking location. The determining module is further configured to: Based on the tunnel video stream, determine the first pose change of the vehicle when leaving the tunnel relative to when entering the tunnel; Based on the information collected by the inertial measurement unit, the second pose change of the vehicle when leaving the tunnel relative to when entering the tunnel is determined; The first pose change and the second pose change are fused to determine the second floor information when the vehicle leaves the passageway.

7. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, cause the electronic device to perform the method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method as described in any one of claims 1-5.

Images