Positioning method, method for generating visual map and apparatus therefor

Abstract

The disclosure provides a positioning method, a visual map generation method and a device thereof, and relates to the technical fields of computer vision, optical character recognition, intelligent transportation and augmented reality in the field of artificial intelligence. The specific implementation scheme is: obtaining a parking space number corresponding to a parking space image, obtaining a three-dimensional coordinate and a three-dimensional pose of the parking space number in a world coordinate system according to the parking space number and a visual map, the visual map being generated according to the parking space number, obtaining a first conversion matrix from a camera coordinate system to a parking space number coordinate system, the parking space number coordinate system being constructed according to the parking space number, and determining a three-dimensional coordinate and a three-dimensional pose of the camera in the world coordinate system according to the first conversion matrix, the three-dimensional coordinate and the three-dimensional pose of the parking space number. The positioning is realized by relying on visual features, and is not affected by factors such as environment and texture repetition, thereby enhancing the positioning effect. The method is easy to quickly deploy and land, and has low maintenance cost in the later period.

Description

Technical Field

[0001] This disclosure relates to the fields of computer vision, optical character recognition, intelligent transportation, and augmented reality in the field of artificial intelligence technology, and in particular to a positioning method, a method for generating visual maps, and an apparatus thereof. Background Technology

[0002] Location technology plays an increasingly important role in people's daily lives, with functions such as driving navigation and finding shops relying on it. However, GPS, Bluetooth, and Wi-Fi signals are easily affected by the environment, making stable positioning difficult in weak signal conditions. Visual positioning, as an emerging solution, is gaining popularity and is widely used in scientific research, industry, and other fields. Examples include efficient panoramic navigation systems for robotic vacuum cleaners and VR house viewing systems. Visual positioning technology can be divided into visual map-based positioning and map-free positioning. However, in these technologies, positioning performance is poor in scenes with high texture repetition, visual map construction is easily affected by various environmental factors, positioning operations are costly, and deployment is difficult. Summary of the Invention

[0003] A positioning method, a method for generating visual maps, and an apparatus thereof are provided.

[0004] According to a first aspect, a positioning method is provided, comprising: obtaining a parking space number corresponding to a parking space image; obtaining the three-dimensional coordinates and three-dimensional pose of the parking space number in a world coordinate system based on the parking space number and a visual map, wherein the visual map is generated based on a plan view of the parking space number; obtaining a first transformation matrix from a camera coordinate system to a parking space number coordinate system, wherein the parking space number coordinate system is constructed based on the plan view of the parking space number; and determining the shooting position and shooting pose of the parking space image based on the first transformation matrix, the three-dimensional coordinates of the parking space number, and the three-dimensional pose of the parking space number.

[0005] According to a second aspect, a method for generating a visual map is provided, comprising: determining the three-dimensional coordinates of multiple parking space numbers in a world coordinate system based on a parking space number plan; determining the three-dimensional pose of the multiple parking space numbers in the world coordinate system based on the parking space number plan; and generating the visual map based on the three-dimensional coordinates and the three-dimensional pose of the multiple parking space numbers.

[0006] According to a third aspect, a positioning device is provided, comprising: a first acquisition module for acquiring a parking space number corresponding to a parking space image; a second acquisition module for acquiring the three-dimensional coordinates and three-dimensional pose of the parking space number in a world coordinate system based on the parking space number and a visual map, wherein the visual map is generated based on the parking space number; a third acquisition module for acquiring a first transformation matrix from a camera coordinate system to a parking space number coordinate system, wherein the parking space number coordinate system is constructed based on the parking space number; and a first determination module for determining the three-dimensional coordinates and three-dimensional pose of a camera in the world coordinate system based on the first transformation matrix, the three-dimensional coordinates of the parking space number, and the three-dimensional pose.

[0007] According to a fourth aspect, a visual map generation apparatus is provided, comprising: a second determining module for determining the three-dimensional coordinates of a plurality of parking space numbers in a world coordinate system based on a parking space number plan; a third determining module for determining the three-dimensional pose of the plurality of parking space numbers in the world coordinate system based on the parking space number plan; and a first generating module for generating the visual map based on the three-dimensional coordinates and the three-dimensional pose of the plurality of parking space numbers.

[0008] According to a fifth aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the positioning method of the first aspect of this disclosure, or the visual map generation method of the second aspect of this disclosure.

[0009] According to a sixth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are configured to cause the computer to perform the positioning method according to the first aspect of the present disclosure, or the visual map generation method according to the second aspect of the present disclosure.

[0010] According to a seventh aspect, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of the positioning method according to the first aspect of this disclosure, or the steps of the visual map generation method according to the second aspect of this disclosure.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0013] Figure 1 This is a flowchart illustrating the positioning method according to the first embodiment of this disclosure;

[0014] Figure 2 This is a schematic diagram of a parking space image in an embodiment of this disclosure;

[0015] Figure 3 This is a flowchart illustrating the positioning method according to the second embodiment of the present disclosure;

[0016] Figure 4 This is a schematic diagram of a scenario where a camera is positioned at a preset location corresponding to a parking space number, as described in this embodiment of the disclosure.

[0017] Figure 5 This is a flowchart illustrating the positioning method according to the third embodiment of this disclosure;

[0018] Figure 6 This is a flowchart illustrating the positioning method according to the fourth embodiment of the present disclosure.

[0019] Figure 7 This is a schematic diagram of the parking space numbering plan in an embodiment of this disclosure;

[0020] Figure 8 This is a schematic diagram of the parking space number coordinate system in an embodiment of this disclosure;

[0021] Figure 9 This is a flowchart illustrating a method for generating a visual map according to a first embodiment of the present disclosure;

[0022] Figure 10 This is a flowchart illustrating a method for generating a visual map according to a second embodiment of the present disclosure;

[0023] Figure 11 This is a flowchart illustrating a method for generating a visual map according to a third embodiment of the present disclosure;

[0024] Figure 12 This is a flowchart illustrating a method for generating a visual map according to a fourth embodiment of the present disclosure;

[0025] Figure 13 This is a block diagram of a positioning device according to a first embodiment of the present disclosure;

[0026] Figure 14 This is a block diagram of a positioning device according to a second embodiment of the present disclosure;

[0027] Figure 15 This is a block diagram of a visual map generation apparatus according to a first embodiment of the present disclosure;

[0028] Figure 16This is a block diagram of a visual map generation apparatus according to a second embodiment of the present disclosure;

[0029] Figure 17 This is a block diagram of an electronic device used to implement the methods of the embodiments of this disclosure. Detailed Implementation

[0030] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0031] Artificial intelligence (AI) is a technical science that studies and develops theories, methods, technologies, and application systems to simulate, extend, and expand human intelligence. Currently, AI technology has the advantages of high automation, high accuracy, and low cost, and has been widely applied.

[0032] Computer vision, also known as machine vision, is a simulation of biological vision using computers and related equipment. More specifically, it refers to using cameras and computers to replace human eyes in identifying, tracking, and measuring targets, and then performing image processing to make the computer-processed images more suitable for human observation or transmission to instruments for detection.

[0033] Optical Character Recognition (OCR) refers to the process of scanning text documents and then analyzing and processing image files to obtain text and layout information. Key performance indicators for OCR systems include: rejection rate, false recognition rate, recognition speed, user interface friendliness, product stability, ease of use, and feasibility.

[0034] Intelligent Traffic System (ITS), also known as Intelligent Transportation System, effectively integrates advanced science and technology (information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operations research, artificial intelligence, etc.) into transportation, service control, and vehicle manufacturing. It strengthens the connection between vehicles, roads, and users, thereby forming a comprehensive transportation system that ensures safety, improves efficiency, improves the environment, and saves energy.

[0035] Augmented Reality (AR) is a technology that calculates the position and angle of camera images in real time and adds corresponding images. It uses a variety of technical means to overlay computer-generated virtual objects or non-geometric information about real objects onto real-world scenes, thereby enhancing the real world.

[0036] The positioning method, visual map generation method, and apparatus of the present disclosure are described below with reference to the accompanying drawings.

[0037] Figure 1 This is a schematic flowchart of the positioning method according to the first embodiment of the present disclosure.

[0038] like Figure 1 As shown, the positioning method of this embodiment may specifically include the following steps:

[0039] S101, Obtain the parking space number corresponding to the parking space image.

[0040] Specifically, the execution entity of the positioning method in this embodiment can be the positioning device provided in this embodiment. The positioning device can be a hardware device with data processing capabilities and / or the necessary software to drive the hardware device. Optionally, the execution entity may include a workstation, server, computer, user terminal, and other devices. The user terminal includes, but is not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, and vehicle terminals. This embodiment uses a parking lot as an example to illustrate the implementation of the positioning method in this scenario.

[0041] In this embodiment of the disclosure, the parking space number corresponding to the parking space in the image is obtained from the parking space image captured by the user through a camera or other imaging device on a mobile terminal, such as... Figure 2 As shown, a vehicle is parked in a parking space in a parking lot. The parking space number is marked on the end of the parking space closest to the road. The parking space number corresponding to the parking space can be obtained from the parking space image with the parking space number through optical character recognition technology.

[0042] S102, obtain the three-dimensional coordinates and three-dimensional pose of the parking space number in the world coordinate system based on the parking space number and the visual map, which is generated based on the parking space number.

[0043] For a parking lot with multiple parking spaces, the location and other relevant information of the parking spaces can be displayed through the floor plan (such as a CAD drawing). In some embodiments, the relevant information corresponding to the parking space number can be obtained from the floor plan based on the parking space number, and then a visual map can be generated based on the relevant information.

[0044] In some implementations, the 3D coordinates and 3D pose of the parking space number in the world coordinate system are obtained from the parking space number obtained from the parking space image and the visual map. The world coordinate system can be understood as a coordinate system with the Z-axis added to the original coordinate system of the planar view.

[0045] S103, obtain the first transformation matrix from the camera coordinate system to the parking space number coordinate system, which is constructed based on the parking space number.

[0046] In some embodiments, a parking space number coordinate system can be constructed with the coordinates of the parking space number as the origin, and a camera coordinate system can be established with the light spot of the camera capturing the parking space image as the origin. Based on the established camera coordinate system and parking space number coordinate system, a first transformation matrix from the camera coordinate system to the parking space coordinate system is obtained. Therefore, the coordinates of a point in the camera coordinate system in the parking space number coordinate system can be determined according to the first transformation matrix. For example, the coordinates of point P in the camera coordinate system are (X... C Y C Z C ), can be based on (X C Y C Z C Using the first transformation matrix, the coordinates (X, Y, F) of spatial point P in the parking space number coordinate system are calculated. O Y O Z O ).

[0047] S104. Based on the first transformation matrix, the three-dimensional coordinates and three-dimensional pose of the parking space number, determine the three-dimensional coordinates and three-dimensional pose of the camera in the world coordinate system.

[0048] In this embodiment of the disclosure, the three-dimensional coordinates and three-dimensional pose of the camera in the world coordinate system can be calculated based on the first transformation matrix from the camera coordinate system to the parking space number coordinate system, the three-dimensional coordinates and three-dimensional pose of the parking space number in the world coordinate system.

[0049] For example: Based on the parking space number obtained from the parking space image and the visual map, determine the three-dimensional coordinates and three-dimensional pose of the parking space number in the world coordinate system (i.e., the position and orientation of the parking space number in the world coordinate system, represented by the matrix To2w). Based on To2w and the first transformation matrix Tc2o, the three-dimensional coordinates and three-dimensional pose of the camera in the world coordinate system can be obtained, that is, six-degree-of-freedom positioning is achieved: Tc2w = To2w * Tc2o.

[0050] In summary, the positioning method of this disclosure obtains the parking space number corresponding to the parking space image, obtains the three-dimensional coordinates and three-dimensional pose of the parking space number in the world coordinate system based on the parking space number and a visual map (generated based on the parking space number), obtains a first transformation matrix from the camera coordinate system to the parking space number coordinate system (constructed based on the parking space number), and determines the three-dimensional coordinates and three-dimensional pose of the camera in the world coordinate system based on the first transformation matrix, the three-dimensional coordinates and three-dimensional pose of the parking space number. This disclosure does not overly rely on visual features for positioning and is not affected by environmental factors or high texture repetition, thus enhancing the positioning effect. The camera (i.e., the user's location) can be located based on the first transformation matrix from the camera coordinate system to the parking space number coordinate system, the visual map of the parking scene, and the parking space number obtained from the parking space image. This disclosure is easy to deploy and implement quickly, with extremely low maintenance costs, which is conducive to commercial mass deployment.

[0051] Figure 3 This is a schematic flowchart of the positioning method according to the second embodiment of the present disclosure.

[0052] like Figure 3 As shown, in Figure 1 Based on the illustrated embodiments, the positioning method of this disclosure may specifically include the following steps:

[0053] S301, Obtain the parking space number corresponding to the parking space image.

[0054] S302: Obtain the three-dimensional coordinates and three-dimensional pose of the parking space number in the world coordinate system based on the parking space number and the visual map, which is generated based on the parking space number.

[0055] Specifically, step S103 in the above embodiments may include the following steps S303-S304:

[0056] S303, obtain the rotation matrix from the camera coordinate system to the parking space number coordinate system.

[0057] In this embodiment, based on the rotation matrix from the camera coordinate system to the parking space number coordinate system, the principal axis of the camera coordinate system can be rotated to the direction of the corresponding principal axis of the parking space number coordinate system. Therefore, by recording the angles that each principal axis of the camera coordinate system needs to rotate when transforming from the camera coordinate system to the parking space number coordinate system, a rotation matrix can be constructed based on multiple rotation angles. The rotation matrix can be used as the rotation component from the camera coordinate system to the parking space number coordinate system.

[0058] In some embodiments, for a vector in the camera coordinate system, the direction of the vector in the parking space number coordinate system can be obtained from the rotation matrix (denoted by R).

[0059] S304. Determine the first transformation matrix based on the preset position vector from the camera to the parking space number and the rotation matrix.

[0060] In this embodiment of the disclosure, the position vector from the camera to the parking space number can be preset as needed. The position vector can be used as a translation component from the camera coordinate system to the parking space number coordinate system. For example, the camera coordinate system and the parking space number coordinate system can be moved according to the position vector so that the origins of the two coordinate systems coincide.

[0061] For example, such as Figure 4 As shown, the preset position of the camera relative to the parking space number is set as follows: the height of the camera (point A) from the parking space number (point B) is 1.5m, the yaw angle relative to the parking space number is 0, and the camera's projection on the ground is located at a position -3m along the y-axis of the parking space number coordinate system. We obtain the position vector T from the camera to the parking space number. The position vector can also be understood as the offset vector.

[0062] In some embodiments, a first transformation matrix from the camera coordinate system to the parking space number coordinate system is determined based on the position vector T and the rotation matrix R.

[0063] S305, based on the first transformation matrix, the three-dimensional coordinates and three-dimensional pose of the parking space number, determine the three-dimensional coordinates and three-dimensional pose of the camera in the world coordinate system.

[0064] Specifically, steps S301-S302 are the same as S101-S102 in the above embodiment, and step S305 is the same as S104 in the above embodiment, which will not be repeated here.

[0065] Furthermore, based on any of the above embodiments, such as Figure 5 As shown, step S303, "obtaining the rotation matrix from the camera coordinate system to the parking space number coordinate system," can specifically include the following steps:

[0066] S501, obtain the first direction of gravity in the camera coordinate system.

[0067] In this embodiment of the disclosure, the direction of gravity in the camera coordinate system, i.e., the first direction, can be obtained by the accelerometer in the mobile device. The first direction is represented by vector V1 in the camera coordinate system.

[0068] S502, obtain the second direction of gravity in the parking space number coordinate system.

[0069] In this embodiment of the disclosure, the direction of gravity in the coordinate system of parking space number is calculated, which is the second direction. The second direction is represented by vector V2 in the parking space number coordinate system.

[0070] S503, determine the rotation matrix based on the first direction and the second direction.

[0071] In this embodiment of the disclosure, the rotation matrix from the camera coordinate system to the parking space number coordinate system is calculated based on the vector V1 corresponding to the first direction and the vector V2 corresponding to the second direction, that is, the rotation matrix from the vector V1 in the camera coordinate system to the vector V2 in the parking space number coordinate system is calculated.

[0072] Therefore, based on the rotation matrix from the camera coordinate system to the parking space number coordinate system and the preset position vector, the first transformation matrix between the two coordinate systems can be obtained, thereby enabling the determination of the camera's coordinates in the parking space coordinate system based on the first transformation matrix.

[0073] As a feasible implementation method, the visual map in the above embodiments can be generated based on the floor plan information corresponding to the parking space number in the parking space number floor plan.

[0074] Based on the above embodiments, such as Figure 6 As shown, the positioning method in this embodiment may further include a visual map generation process, which may specifically include the following steps:

[0075] S601, determine the three-dimensional coordinates of multiple parking space numbers in the world coordinate system based on the parking space numbering plan.

[0076] In this embodiment of the disclosure, the origin of the plan view is used as the origin of the world coordinate system to establish a world coordinate system, and the three-dimensional coordinates of multiple parking space numbers in the parking space number plan view are determined based on the coordinate system.

[0077] As a feasible implementation method, when constructing a visual map of a parking lot on the ground, it is only necessary to determine the two-dimensional coordinates (x, y) of the parking space number in the coordinate system of the origin of the plan view. Based on this, the three-dimensional coordinates of the parking space number in the world coordinate system can be obtained by padding the Z-axis with 0. If it is an underground parking lot, the coordinate value of the Z-axis can be determined according to the actual distance of the parking lot from the ground.

[0078] In some embodiments, the two-dimensional coordinates of the parking space number in the coordinate system of the origin of the floor plan can be determined in the following way: (e.g.) Figure 7 As shown, in the parking space numbering plan, each parking space number corresponds to a parking space border. For each parking space number, the two-dimensional coordinates of the two corner points of the corresponding parking space border are obtained (e.g., ...). Figure 7 The two corner points of the parking space frame corresponding to parking space No. 037 are adjacent to the road (corner point C and corner point D). The two-dimensional coordinates of the midpoint of the line segment between the two corner points are obtained based on the two-dimensional coordinates of the two corner points. These two-dimensional coordinates are then used as the two-dimensional coordinates of the parking space number corresponding to the parking space frame.

[0079] S602, determine the three-dimensional pose of multiple parking space numbers in the world coordinate system based on the parking space numbering plan.

[0080] In some embodiments, a parking space number coordinate system corresponding to the parking space number is constructed based on the parking space number plan, and the three-dimensional pose of the parking space number in the world coordinate system is determined based on the parking space number coordinate system and the world coordinate system.

[0081] For example, such as Figure 8 As shown, a coordinate system for parking space numbers is established with the coordinate point of the parking space number as the origin, the x-axis as the direction parallel to the road and facing the parking space number from left to right, the y-axis as the direction from the side of the parking space closer to the road to the side farther from the road, and the z-axis as the direction from the ground to the sky. Figure 8 The coordinate systems corresponding to parking space number 007 and parking space number 037 are defined. For multiple parking space numbers, the translation and rotation components from the corresponding parking space number coordinate systems to the world coordinate system are determined. Based on the translation and rotation components, the three-dimensional pose of the multiple parking space numbers in the world coordinate system is determined, i.e., the orientation of the parking space numbers in the world coordinate system.

[0082] S603 generates a visual map based on the three-dimensional coordinates and three-dimensional poses of multiple parking space numbers.

[0083] In this embodiment of the disclosure, a mapping relationship between parking space numbers and their corresponding three-dimensional coordinates and poses is formed based on the three-dimensional coordinates and poses of multiple parking space numbers. The location and orientation information corresponding to the multiple parking space numbers are then summarized to form a visual map.

[0084] Therefore, based on the parking space information corresponding to each parking space number in the parking space number plan, such as the positions of the two corner points of the parking space border and the arrangement and distribution information of each parking space, a parking space number coordinate system is constructed. The parking space number coordinate system and the world coordinate system are determined to determine the three-dimensional pose of the parking space number in the world coordinate system, thereby enhancing the degree of freedom of positioning and providing more accurate positioning information.

[0085] In summary, the positioning method of this disclosure obtains the parking space number corresponding to the parking space image, obtains the three-dimensional coordinates and three-dimensional pose of the parking space number in the world coordinate system based on the parking space number and a visual map (generated based on the parking space number), obtains a first transformation matrix from the camera coordinate system to the parking space number coordinate system (constructed based on the parking space number), and determines the three-dimensional coordinates and three-dimensional pose of the camera in the world coordinate system based on the first transformation matrix, the three-dimensional coordinates and three-dimensional pose of the parking space number. This disclosure does not overly rely on visual features for positioning and is not affected by environmental factors or high texture repetition, thus enhancing the positioning effect. The first transformation matrix from the camera coordinate system to the parking space number coordinate system is determined based on a preset position vector from the camera to the parking space number and a rotation matrix from the camera coordinate system to the parking space number coordinate system. Based on the first transformation matrix, the visual map of the parking lot, and the parking space number obtained from the parking space image, the camera (i.e., the user's location) can be located. This method is easy to deploy and implement quickly, with extremely low maintenance costs, facilitating large-scale commercial deployment.

[0086] To implement the above embodiments, this disclosure also provides a method for generating visual maps. Figure 9 This is a schematic flowchart of a method for generating a visual map according to a first embodiment of the present disclosure.

[0087] like Figure 9 As shown, the method for generating a visual map according to embodiments of this disclosure may specifically include the following steps:

[0088] S901, determine the three-dimensional coordinates of multiple parking space numbers in the world coordinate system based on the parking space numbering plan.

[0089] Specifically, the execution entity of the visual map generation method in this disclosure embodiment can be the visual map generation apparatus provided in this disclosure embodiment. This generation apparatus can be a hardware device with data processing capabilities and / or the necessary software to drive the hardware device. Optionally, the execution entity may include a workstation, server, computer, user terminal, and other devices. The user terminal includes, but is not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, and vehicle terminals.

[0090] S902, determine the three-dimensional pose of multiple parking space numbers in the world coordinate system based on the parking space numbering plan.

[0091] S903 generates a visual map based on the three-dimensional coordinates and three-dimensional poses of multiple parking space numbers.

[0092] It should be noted that the method for generating the visual map in this embodiment is the same as that in the above embodiments, and will not be repeated here.

[0093] In summary, the visual map generation method of this disclosure determines the three-dimensional coordinates of multiple parking space numbers in the world coordinate system based on the parking space number plan view, determines the three-dimensional pose of multiple parking space numbers in the world coordinate system based on the parking space number plan view, and generates a visual map based on the three-dimensional coordinates and three-dimensional pose of the multiple parking space numbers. This disclosure generates a visual map based on the parking space number plan view, eliminating the need for extensive data and image collection of the parking scene, thus saving costs, and avoiding the impact of environmental factors such as lighting on visual map construction.

[0094] Figure 10 This is a schematic flowchart of a method for generating a visual map according to a second embodiment of the present disclosure.

[0095] like Figure 10 As shown, based on the above embodiments, the visual map generation method of this disclosure specifically includes the following steps:

[0096] The above step S901 may specifically include steps S1001-S1002:

[0097] S1001, Obtain the two-dimensional coordinates of the parking space number in the origin coordinate system of the parking space number plan based on the parking space number plan.

[0098] S1002, determine the three-dimensional coordinates of the parking space number in the world coordinate system based on the two-dimensional coordinates of the parking space number.

[0099] S1003, determine the three-dimensional pose of multiple parking space numbers in the world coordinate system based on the parking space numbering plan.

[0100] S1004 generates a visual map based on the three-dimensional coordinates and three-dimensional poses of multiple parking space numbers.

[0101] Specifically, steps S1003-S1004 are the same as S902-S903 in the above embodiments. The method for generating the visual map in this embodiment is the same as in the above embodiments, and will not be repeated here.

[0102] Furthermore, based on the above embodiments, such as Figure 11 As shown, step S1001, "obtaining the two-dimensional coordinates of the parking space number in the coordinate system of the origin of the parking space number plan," may specifically include the following steps:

[0103] S1101, Obtain the two-dimensional coordinates of the two corner points of the parking space frame based on the parking space number plan.

[0104] S1102, determine the two-dimensional coordinates of the parking space number corresponding to the parking space border based on the two-dimensional coordinates of the two corner points.

[0105] It should be noted that the method for generating the visual map in this embodiment is the same as that in the above embodiments, and will not be repeated here.

[0106] Furthermore, based on the above embodiments, such as Figure 12 As shown, step S1003, "determining the three-dimensional pose of multiple parking space numbers in the world coordinate system based on the parking space number plan," may specifically include the following steps:

[0107] S1201, Construct a coordinate system for the parking space numbers corresponding to the parking space numbers based on the parking space number plan.

[0108] S1202, determine the three-dimensional pose of the parking space number in the world coordinate system based on the parking space number coordinate system and the world coordinate system.

[0109] It should be noted that the method for generating the visual map in this embodiment is the same as that in the above embodiments, and will not be repeated here.

[0110] In summary, the visual map generation method of this disclosure determines the three-dimensional coordinates of multiple parking space numbers in the world coordinate system based on the parking space number plan view, determines the three-dimensional pose of multiple parking space numbers in the world coordinate system based on the parking space number plan view, and generates a visual map based on the three-dimensional coordinates and three-dimensional pose of the multiple parking space numbers. This disclosure generates a visual map based on the parking space number plan view, eliminating the need for extensive data and image collection of the parking scene, thus saving costs, and avoiding the impact of environmental factors such as lighting on visual map construction.

[0111] Figure 13 This is a block diagram of a positioning device according to a first embodiment of the present disclosure.

[0112] like Figure 13 As shown, the positioning device 1300 of this embodiment includes: a first acquisition module 1301, a second acquisition module 1302, a third acquisition module 1303, and a first determination module 1304.

[0113] The first acquisition module 1301 is used to acquire the parking space number corresponding to the parking space image.

[0114] The second acquisition module 1302 is used to acquire the three-dimensional coordinates and three-dimensional pose of the parking space number in the world coordinate system based on the parking space number and the visual map, which is generated based on the parking space number.

[0115] The third acquisition module 1303 is used to acquire the first transformation matrix from the camera coordinate system to the parking space number coordinate system, which is constructed based on the parking space number.

[0116] The first determining module 1304 is used to determine the three-dimensional coordinates and three-dimensional pose of the camera in the world coordinate system based on the first transformation matrix, the three-dimensional coordinates and three-dimensional pose of the parking space number.

[0117] It should be noted that the above explanation of the positioning method embodiments also applies to the positioning device of the present disclosure embodiments, and the specific process will not be repeated here.

[0118] In summary, the positioning device of this embodiment obtains the parking space number corresponding to the parking space image, obtains the three-dimensional coordinates and three-dimensional pose of the parking space number in the world coordinate system based on the parking space number and a visual map (generated based on the parking space number), obtains a first transformation matrix from the camera coordinate system to the parking space number coordinate system (constructed based on the parking space number), and determines the three-dimensional coordinates and three-dimensional pose of the camera in the world coordinate system based on the first transformation matrix, the three-dimensional coordinates and three-dimensional pose of the parking space number. This disclosure does not overly rely on visual features for positioning and is not affected by environmental factors or high texture repetition, thus enhancing the positioning effect. The camera (i.e., the user's location) can be located based on the first transformation matrix from the camera coordinate system to the parking space number coordinate system, the visual map of the parking scene, and the parking space number obtained from the parking space image. This disclosure is easy to deploy and implement quickly, with extremely low maintenance costs, which is conducive to commercial mass deployment.

[0119] Figure 14 This is a block diagram of a positioning device according to a second embodiment of the present disclosure.

[0120] like Figure 14 As shown, the positioning device 1400 of this embodiment includes: a first acquisition module 1401, a second acquisition module 1402, a third acquisition module 1403, and a first determination module 1404.

[0121] The first acquisition module 1401 has the same structure and function as the first acquisition module 1301 in the previous embodiment, the second acquisition module 1402 has the same structure and function as the second acquisition module 1302 in the previous embodiment, the third acquisition module 1403 has the same structure and function as the third acquisition module 1303 in the previous embodiment, and the first determination module 1404 has the same structure and function as the first determination module 1304 in the previous embodiment.

[0122] Furthermore, the third acquisition module 1403 may include: a first acquisition unit 14031, used to acquire the rotation matrix from the camera coordinate system to the parking space number coordinate system; and a first determination unit 14032, used to determine a first transformation matrix based on the preset position vector and rotation matrix from the camera to the parking space number.

[0123] Furthermore, the first acquisition unit includes: a first acquisition subunit for acquiring the first direction of gravity in the camera coordinate system; a second acquisition subunit for acquiring the second direction of gravity in the parking space number coordinate system; and a first determination subunit for determining a rotation matrix based on the first and second directions.

[0124] Furthermore, the first acquisition module 1401 may include a detection unit for performing optical character recognition detection on the parking space image to obtain the parking space number.

[0125] Furthermore, the visual map is generated based on the floor plan information corresponding to the parking space number in the parking space number floor plan.

[0126] Furthermore, the positioning device 1400 may also include: a fourth determining module for determining the three-dimensional coordinates of multiple parking space numbers in the world coordinate system based on the parking space number plan; a fifth determining module for determining the three-dimensional pose of multiple parking space numbers in the world coordinate system based on the parking space number plan; and a second generating module for generating a visual map based on the three-dimensional coordinates and three-dimensional pose of the multiple parking space numbers.

[0127] Furthermore, the fourth determining module includes: a second obtaining unit, used to obtain the two-dimensional coordinates of the parking space number in the origin coordinate system of the parking space number plan based on the parking space number plan; and a second determining unit, used to determine the three-dimensional coordinates of the parking space number in the world coordinate system based on the two-dimensional coordinates of the parking space number.

[0128] Furthermore, the second acquisition unit includes: a third acquisition subunit, used to acquire the two-dimensional coordinates of two corner points of the parking space frame according to the parking space number plan; and a second determination subunit, used to determine the two-dimensional coordinates of the parking space number corresponding to the parking space frame according to the two-dimensional coordinates of the two corner points.

[0129] Furthermore, the fifth determining module includes: a first construction unit, used to construct a parking space number coordinate system corresponding to the parking space number based on the parking space number plan; and a third determining unit, used to determine the three-dimensional pose of the parking space number in the world coordinate system based on the parking space number coordinate system and the world coordinate system.

[0130] In summary, the positioning device of this embodiment obtains the parking space number corresponding to the parking space image, obtains the three-dimensional coordinates and three-dimensional pose of the parking space number in the world coordinate system based on the parking space number and a visual map (generated based on the parking space number), obtains a first transformation matrix from the camera coordinate system to the parking space number coordinate system (constructed based on the parking space number), and determines the three-dimensional coordinates and three-dimensional pose of the camera in the world coordinate system based on the first transformation matrix, the three-dimensional coordinates and three-dimensional pose of the parking space number. This disclosure does not overly rely on visual features for positioning and is not affected by environmental factors or high texture repetition, thus enhancing the positioning effect. The first transformation matrix from the camera coordinate system to the parking space number coordinate system is determined based on a preset position vector from the camera to the parking space number and a rotation matrix from the camera coordinate system to the parking space number coordinate system. Based on the first transformation matrix, the visual map of the parking lot, and the parking space number obtained from the parking space image, the camera (i.e., the user's location) can be located. This method is easy to deploy and implement quickly, with extremely low maintenance costs, facilitating large-scale commercial deployment.

[0131] Figure 15 This is a block diagram of a visual map generation apparatus according to a first embodiment of the present disclosure.

[0132] like Figure 15 As shown, the visual map generation apparatus 1500 of this embodiment includes: a second determining module 1501, a third determining module 1502 and a first generating module 1503.

[0133] The second determining module 1501 is used to determine the three-dimensional coordinates of multiple parking space numbers in the world coordinate system based on the parking space numbering plan.

[0134] The third determining module 1502 is used to determine the three-dimensional pose of multiple parking space numbers in the world coordinate system based on the parking space number plan.

[0135] The first generation module 1503 is used to generate a visual map based on the three-dimensional coordinates and three-dimensional poses of multiple parking space numbers.

[0136] It should be noted that the above explanation of the embodiments of the visual map generation method also applies to the visual map generation apparatus of the embodiments of this disclosure, and the specific process will not be repeated here.

[0137] In summary, the visual map generation apparatus of this disclosure determines the three-dimensional coordinates of multiple parking space numbers in the world coordinate system based on the parking space number plan view, determines the three-dimensional pose of multiple parking space numbers in the world coordinate system based on the parking space number plan view, and generates a visual map based on the three-dimensional coordinates and three-dimensional pose of the multiple parking space numbers. This disclosure generates a visual map based on the parking space number plan view, eliminating the need for extensive data and image collection of the parking scene, thus saving costs, and avoiding the influence of environmental factors such as lighting on the visual map construction.

[0138] Figure 16 This is a block diagram of a visual map generation apparatus according to a second embodiment of the present disclosure.

[0139] like Figure 16 As shown, the visual map generation apparatus 1600 of this embodiment includes: a second determining module 1601, a third determining module 1602 and a first generating module 1603.

[0140] The second determining module 1601 has the same structure and function as the second determining module 1501 in the previous embodiment, the third determining module 1602 has the same structure and function as the third determining module 1502 in the previous embodiment, and the first generating module 1603 has the same structure and function as the first generating module 1503 in the previous embodiment.

[0141] Furthermore, the second determining module 1601 includes: a third obtaining unit 16011, used to obtain the two-dimensional coordinates of the parking space number in the origin coordinate system of the parking space number plan based on the parking space number plan; and a fourth determining unit 16012, used to determine the three-dimensional coordinates of the parking space number in the world coordinate system based on the two-dimensional coordinates of the parking space number.

[0142] Furthermore, the third acquisition unit 16011 includes: a fourth acquisition subunit, used to acquire the two-dimensional coordinates of two corner points of the parking space frame according to the parking space number plan; and a third determination subunit, used to determine the two-dimensional coordinates of the parking space number corresponding to the parking space frame according to the two-dimensional coordinates of the two corner points.

[0143] Furthermore, the third determining module 1602 includes: a second constructing unit 16021, used to construct a parking space number coordinate system corresponding to the parking space number based on the parking space number plan; and a fifth determining unit 16022, used to determine the three-dimensional pose of the parking space number in the world coordinate system based on the parking space number coordinate system and the world coordinate system.

[0144] In summary, the visual map generation apparatus of this disclosure determines the three-dimensional coordinates of multiple parking space numbers in the world coordinate system based on the parking space number plan view, determines the three-dimensional pose of multiple parking space numbers in the world coordinate system based on the parking space number plan view, and generates a visual map based on the three-dimensional coordinates and three-dimensional pose of the multiple parking space numbers. This disclosure generates a visual map based on the parking space number plan view, eliminating the need for extensive data and image collection of the parking scene, thus saving costs, and avoiding the influence of environmental factors such as lighting on the visual map construction.

[0145] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0146] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0147] Figure 17 A schematic block diagram of an example electronic device 1700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0148] like Figure 17 As shown, the electronic device 1700 includes a computing unit 1701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1702 or a computer program loaded into a random access memory (RAM) 1703 from a storage unit 1708. The RAM 1703 may also store various programs and data required for the operation of the electronic device 1700. The computing unit 1701, ROM 1702, and RAM 1703 are interconnected via a bus 1704. An input / output (I / O) interface 1705 is also connected to the bus 1704.

[0149] Multiple components in electronic device 1700 are connected to I / O interface 1705, including: input unit 1706, such as keyboard, mouse, etc.; output unit 1707, such as various types of displays, speakers, etc.; storage unit 1708, such as disk, optical disk, etc.; and communication unit 1709, such as network card, modem, wireless transceiver, etc. Communication unit 1709 allows electronic device 1700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0150] Computing unit 1701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing unit 1701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Computing unit 1701 performs the various methods and processes described above, such as... Figures 1 to 8 The positioning method shown, or Figures 9 to 12 The method for generating a visual map is illustrated. For example, in some embodiments, the positioning method or the method for generating a visual map may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1708. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 1700 via ROM 1702 and / or communication unit 1709. When the computer program is loaded into RAM 1703 and executed by computing unit 1701, one or more steps of the positioning method or the method for generating a visual map described above may be performed. Alternatively, in other embodiments, computing unit 1701 may be configured to perform the positioning method or the method for generating a visual map by any other suitable means (e.g., by means of firmware).

[0151] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0152] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0153] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0154] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0155] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0156] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0157] According to embodiments of this disclosure, this disclosure also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the steps of the positioning method or the steps of the visual map generation method shown in the above embodiments of this disclosure.

[0158] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0159] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for generating a visual map, comprising: Determine the three-dimensional coordinates of multiple parking space numbers in the world coordinate system based on the parking space numbering floor plan; Determine the three-dimensional pose of the multiple parking space numbers in the world coordinate system based on the parking space numbering plan; as well as Based on the three-dimensional coordinates and three-dimensional poses of the plurality of parking space numbers, a mapping relationship is formed between the plurality of parking space numbers and their corresponding three-dimensional coordinates and three-dimensional poses, and the visual map is generated based on the mapping relationship; The step of determining the three-dimensional pose of the plurality of parking space numbers in the world coordinate system based on the parking space number plan includes: For any of the aforementioned parking space numbers, construct a parking space number coordinate system corresponding to the parking space number based on the parking space number plan; and Based on the parking space number coordinate system and the world coordinate system, determine the three-dimensional pose of the parking space number in the world coordinate system; The step of constructing a parking space number coordinate system corresponding to the parking space number based on the parking space number plan includes: Based on the parking space number plan, obtain the two-dimensional coordinates of the two corner points of the parking space frame corresponding to the parking space number, and determine the coordinate point of the parking space number in the plan coordinate system based on the two-dimensional coordinates of the two corner points; A coordinate system for the parking space number is established with the coordinate point of the parking space number as the origin, the x-axis as the direction parallel to the road and facing the parking space number from left to right, the y-axis as the direction from the side of the parking space closer to the road to the side farther away from the road, and the z-axis as the direction from the ground to the sky.

2. The generation method according to claim 1, wherein, The step of determining the three-dimensional coordinates of multiple parking space numbers in the world coordinate system based on the parking space numbering plan includes: Based on the parking space numbering plan, obtain the two-dimensional coordinates of the parking space number in the coordinate system of the origin of the plan; and The three-dimensional coordinates of the parking space number in the world coordinate system are determined based on the two-dimensional coordinates of the parking space number.

3. The generation method according to claim 2, wherein, The step of obtaining the two-dimensional coordinates of the parking space number in the origin coordinate system of the parking space number plan includes: Based on the parking space numbering plan, obtain the two-dimensional coordinates of the two corner points of the parking space frame; and The two-dimensional coordinates of the parking space number corresponding to the parking space border are determined based on the two-dimensional coordinates of the two corner points.

4. A visual map generation apparatus, comprising: The second determining module is used to determine the three-dimensional coordinates of multiple parking space numbers in the world coordinate system based on the parking space numbering plan; The third determining module is used to determine the three-dimensional pose of the plurality of parking space numbers in the world coordinate system based on the parking space number plan; as well as The first generation module is used to form a mapping relationship between the multiple parking space numbers and their corresponding three-dimensional coordinates and three-dimensional poses based on the three-dimensional coordinates and three-dimensional poses of the multiple parking space numbers, and to generate the visual map based on the mapping relationship. The third determining module includes: The second construction unit is used to construct a parking space number coordinate system corresponding to any of the parking space numbers based on the parking space number plan. The fifth determining unit is used to determine the three-dimensional pose of the parking space number in the world coordinate system based on the parking space number coordinate system and the world coordinate system. The second building unit is further configured to: Based on the parking space number plan, obtain the two-dimensional coordinates of the two corner points of the parking space frame corresponding to the parking space number, and determine the coordinate point of the parking space number in the plan coordinate system based on the two-dimensional coordinates of the two corner points; A coordinate system for the parking space number is established with the coordinate point of the parking space number as the origin, the x-axis as the direction parallel to the road and facing the parking space number from left to right, the y-axis as the direction from the side of the parking space closer to the road to the side farther away from the road, and the z-axis as the direction from the ground to the sky.

5. The apparatus of claim 4, wherein, The second determining module includes: The third acquisition unit is used to obtain the two-dimensional coordinates of the parking space number in the original coordinate system of the parking space number plan based on the parking space number plan. The fourth determining unit is used to determine the three-dimensional coordinates of the parking space number in the world coordinate system based on the two-dimensional coordinates of the parking space number.

6. The apparatus of claim 5, wherein, The third acquisition unit includes: The fourth acquisition subunit is used to obtain the two-dimensional coordinates of the two corner points of the parking space frame based on the parking space number plan; the third determination subunit is used to determine the two-dimensional coordinates of the parking space number corresponding to the parking space frame based on the two-dimensional coordinates of the two corner points.

7. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-3.

8. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-3.

9. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-3.

Images