System and method for distance measurement

Abstract

A system (100) for distance measurement includes at least one storage device (140) and at least one processor (122). The storage device (140) includes a set of instructions. The at least one processor (122) communicates with the at least one storage device (140). When executing the set of instructions, the at least one processor (122) is configured to cause the system (100) to perform steps 501-505. Step 501, acquiring an image captured by a camera, the image including an object. Step 503, based on the object, identifying at least one target line in the image, wherein the at least one target line is associated with at least two preset reference lines, each of the at least two preset reference lines corresponding to a horizontal reference distance from the camera. Step 505, determining a distance between the camera and the object based on the at least one target line.

Description

Technical Field

[0001] The present application relates generally to systems and methods for distance measurement and, more particularly, to systems and methods for determining the distance between an object and a camera mounted on a vehicle. Background Art

[0002] With the development of microelectronics and robotics, the exploration of unmanned driving has developed rapidly. Generally, an autonomous driving system can sense environmental information through one or more sensor devices (e.g., cameras, lidars). For example, a camera (e.g., a monocular camera) can be used to determine the distance between an object (e.g., a pedestrian, a vehicle) and an autonomous driving vehicle. However, the camera is mainly used to determine the distance between the object and the vehicle when the vehicle is traveling on a flat road, that is, the height information of the road is not taken into account, which may result in inaccurate distance measurements when the vehicle is traveling on a non-flat road (e.g., a slope, an undulating road). Therefore, it is desirable to provide a system and method for effectively and accurately determining the distance between a vehicle and an object. Summary of the invention

[0003] An embodiment of the present application provides a system for distance measurement. The exemplary system includes at least one storage device, the at least one storage device includes a set of instructions, and at least one processor, the at least one processor communicating with the at least one storage device. When executing the set of instructions, the at least one processor is configured to enable the system to acquire an image captured by a camera, the image including an object; based on the object, identify at least one target line in the image, wherein the at least one target line is associated with at least two preset reference lines, each of the at least two preset reference lines corresponding to a horizontal reference distance from the camera; and based on the at least one target line, determine the distance between the camera and the object.

[0004] The embodiment of the present application further provides a method for distance measurement. The exemplary method is implemented on a computing device, the computing device including at least one processor, at least one storage device, and a communication platform connected to a network, the method comprising acquiring an image captured by a camera, the image including an object; based on the object, identifying at least one target line in the image, wherein the at least one target line is associated with at least two preset reference lines, each of the at least two preset reference lines corresponding to a horizontal reference distance from the camera; and determining the distance between the camera and the object based on the at least one target line.

[0005] The embodiment of the present application also provides another system for distance measurement. The exemplary system includes an acquisition module, which is configured to acquire an image captured by a camera, the image including an object; a target line recognition module, which is configured to recognize at least one target line in the image based on the object, wherein the at least one target line is associated with at least two preset reference lines, each of the at least two preset reference lines respectively corresponding to a horizontal reference distance from the camera to the camera; and a distance determination module, which is configured to determine the distance between the camera and the object based on the at least one target line.

[0006] An embodiment of the present application also provides a non-transitory computer-readable storage medium for distance measurement. The exemplary non-transitory computer-readable storage medium includes at least one set of instructions, wherein when executed by at least one processor of a computing device, the at least one set of instructions instructs the at least one processor to perform the following actions: acquiring an image captured by a camera, the image including an object; based on the object, identifying at least one target line in the image, wherein the at least one target line is associated with at least two preset reference lines, each of the at least two preset reference lines corresponding to a horizontal reference distance from the camera; and determining the distance between the camera and the object based on the at least one target line.

[0007] The embodiment of the present application also provides an autonomous driving vehicle for distance measurement. The autonomous driving vehicle for distance measurement includes a detection component, a planning component, and a control component, wherein the planning component is configured to obtain an image captured by a camera, the image including an object; based on the object, identify at least one target line in the image, wherein the at least one target line is associated with at least two preset reference lines, each of the at least two preset reference lines corresponding to a horizontal reference distance from the camera; and based on the at least one target line, determine the distance between the camera and the object.

[0008] Some additional features of the present application may be explained in the following description. Some additional features of the present application will be apparent to those skilled in the art through study of the following description and corresponding drawings or understanding of the production or operation of the embodiments. Features of the present application may be realized and obtained by practicing or using various aspects of the methods, means and combinations set forth in the detailed examples discussed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The present application will be further described by exemplary embodiments. These exemplary embodiments will be described in detail by the accompanying drawings. These embodiments are non-limiting exemplary embodiments, wherein throughout all views of the accompanying drawings, like reference numerals represent like structures, wherein:

[0010] Figure 1is a schematic diagram of an exemplary autonomous driving system according to some embodiments of the present application;

[0011] Figure 2 is a schematic diagram of exemplary hardware and / or software components of an exemplary computing device according to some embodiments of the present application;

[0012] Figure 3 is a schematic diagram of exemplary hardware and / or software components of an exemplary mobile device according to some embodiments of the present application;

[0013] Figure 4 is a block diagram of an exemplary processing device according to some embodiments of the present application;

[0014] Figure 5 is a flowchart of an exemplary process for determining a distance between a camera and an object according to some embodiments of the present application;

[0015] Fig. 6A and Figure 6B is a schematic diagram of a positional relationship between a camera and at least two preset reference lines according to some embodiments of the present application;

[0016] Figure 6C is a schematic diagram of an exemplary prediction line corresponding to at least two preset reference lines according to some embodiments of the present application;

[0017] Figures 7A-7E is a schematic diagram of exemplary measurement points associated with an object according to some embodiments of the present application;

[0018] Figures 8A-8F is a schematic diagram of an exemplary process for determining at least one target line based on at least one measurement point according to some embodiments of the present application;

[0019] Fig. 9 is a flowchart of an exemplary process for identifying at least one target line according to some embodiments of the present application;

[0020] Fig.10 is a schematic diagram of an exemplary process for projecting a preset reference line onto an image according to some embodiments of the present application;

[0021] Fig.11 is a flowchart of an exemplary process for determining a distance between a camera and an object according to some embodiments of the present application;

[0022] Fig. 12A is a schematic diagram of a side view of an exemplary distance between a camera and an object according to some embodiments of the present application; and

[0023] Fig. 12B is a schematic diagram of a side view of an exemplary distance between a camera and an object according to some embodiments of the present application. DETAILED DESCRIPTION

[0024] The following description is intended to enable one of ordinary skill in the art to implement and utilize the present application, and the description is provided in the context of a specific application scenario and its requirements. It will be apparent to one of ordinary skill in the art that various changes may be made to the disclosed embodiments, and that the general principles defined in the present application may be applied to other embodiments and application scenarios without departing from the principles and scope of the present application. Therefore, the present application is not limited to the described embodiments, but should be given the broadest scope consistent with the claims.

[0025] The terms used in this application are only used to describe specific exemplary embodiments and do not limit the scope of this application. As shown in this application, singular forms such as "one", "a", "a kind of" and / or "the" may also include plural forms unless the context clearly indicates an exception. It should be understood that the terms "including" and "comprising" used in this application only indicate clearly identified features, integers, steps, operations, elements and / or parts, and do not exclude the presence and addition of one or more other features, integers, steps, operations, elements, parts and / or combinations thereof.

[0026] These and other features, characteristics and functions and methods of operation of the related structural elements of the present application, as well as the combination of components and manufacturing economy, may become more apparent from the following description of the accompanying drawings, which form a part of the specification of this application. However, it should be clearly understood that the drawings are for illustration and description purposes only and are not intended to limit the scope of the present application. It should be understood that the drawings are not drawn to scale.

[0027] Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the operations in the flow charts may not be performed in order. Instead, the various steps may be processed in reverse order or simultaneously. At the same time, one or more other operations may also be added to these flow charts. One or more operations may also be deleted from the flow charts.

[0028] In addition, although the systems and methods disclosed in this application are mainly described for land transportation systems, it should be understood that this is only an exemplary embodiment. The system and method of the present application can be any other type of transportation system. For example, the system and method of the present application can be a transportation system for different environments including land, sea, aerospace, etc. or different combinations thereof. The self-driving cars of the transportation system may include taxis, private cars, ride-sharing, buses, trains, motor vehicles, high-speed railways, subways, ships, airplanes, spacecraft, hot air balloons, etc., or any combination thereof.

[0029] One aspect of the present application relates to a system and method for distance measurement, for example, determining the distance between an object and a vehicle (e.g., an autonomous vehicle). The system can acquire an image including an object captured by a camera (e.g., a monocular camera) mounted on a vehicle. The system can also identify at least one target line in the image based on the object. At least one target line can be determined based on at least two preset reference lines, wherein each of the at least two preset reference lines can correspond to a corresponding horizontal reference distance to the camera. For example, the system can determine at least one measurement point associated with the object, and determine at least one target line based on at least one measurement point and at least one prediction line corresponding to at least one of the at least two preset reference lines. In addition, the system can determine the distance between the camera (or vehicle) and the object based on at least one target line. According to the system and method of the present application, at least two preset reference lines are introduced to determine the distance between the camera (or vehicle equipped with a camera) and the object, thereby improving the efficiency and accuracy of distance measurement.

[0030] Figure 1 1 is a schematic diagram of an exemplary autonomous driving system according to some embodiments of the present application. In some embodiments, the autonomous driving system 100 may include a vehicle 110 (e.g., 110-1, 110-2, ..., 110-n), a server 120, a terminal device 130, a storage device 140, a network 150, and a positioning and navigation system 160.

[0031] The vehicle 110 may be any type of autonomous vehicle, drone, etc. As used herein, an autonomous vehicle or drone may refer to a vehicle that is capable of achieving a certain level of driving automation. Exemplary levels of driving automation may include: a first level, at which the vehicle is primarily supervised by a human and has specific autonomous functions (e.g., automatic steering or acceleration); a second level, at which the vehicle has one or more advanced driver assistance systems (ADAS) (e.g., adaptive cruise control system, lane keeping system), which can control the braking, steering and / or acceleration of the vehicle; a third level, at which the vehicle can be controlled to travel automatically when one or more specific conditions are met; a fourth level, at which the vehicle can operate without human input or supervision, but is still subject to certain constraints (e.g., limited to a specific area); a fifth level, at which the vehicle can operate autonomously in all situations or similar situations or any combination thereof.

[0032] In some embodiments, the vehicle 110 may have an equivalent structure that enables the vehicle 110 to move around or fly. For example, the vehicle 110 may include the structure of a conventional vehicle, such as a chassis, a suspension, a steering device (e.g., a steering wheel), a braking device (e.g., a brake pedal), an accelerator, etc. For another example, the vehicle 110 may have a body and at least one wheel. The body may be any style of body, such as a sports car, a coupe, a sedan, a light truck, a station wagon, an off-road vehicle (SUV), a minivan, or a modified car. At least one wheel may be configured as all-wheel drive (AWD), front-wheel drive (FWR), rear-wheel drive (RWD), etc. In some embodiments, the intended vehicle 110 may be an electric vehicle, a fuel cell vehicle, a hybrid vehicle, a conventional internal combustion engine vehicle, etc.

[0033] In some embodiments, the vehicle 110 can sense its environment and navigate through one or more detection units 112. At least two detection units 112 may include radar (e.g., laser radar (LiDAR)), global positioning system (GPS) module, inertial measurement unit (IMU), camera, etc., or any combination thereof. The radar (e.g., LiDAR) can be configured to scan the surroundings of the vehicle 110 and generate point cloud data. The point cloud data can be used to generate a digital three-dimensional (3D) representation of one or more objects around the vehicle 110. The GPS module may refer to a device capable of receiving geographic location and time information from a GPS satellite and then determining the geographic location of the device. IMU may refer to an electronic device that uses various inertial sensors to measure and provide the specific force, angular velocity, and sometimes the magnetic field around the vehicle. In some embodiments, the various inertial sensors may include acceleration sensors (e.g., piezoelectric sensors), speed sensors (e.g., Hall sensors), distance sensors (e.g., radar, infrared sensors), steering angle sensors (e.g., tilt sensors), sensors related to traction (e.g., force sensors), etc. The camera can be configured to acquire one or more images related to an object (e.g., a person, an animal, a tree, a roadblock, a building, or a vehicle) within the camera range. The camera can be a static camera, a camera, etc., or a combination thereof. In some embodiments, the camera can include a monocular camera, a binocular camera, a multi-view camera, etc., or a combination thereof. In some embodiments, the camera can be mounted on the vehicle 110 (e.g., the roof of the vehicle 110). In some embodiments, the camera can be placed horizontally or tilted relative to the vehicle 110.

[0034] In some embodiments, the server 120 may be a single server or a server group. The server group may be centralized or distributed (for example, the server 120 may be a distributed system). In some embodiments, the server 120 may be local or remote. For example, the server 120 may access information and / or data stored in the terminal device 130, the detection unit 112, the vehicle 110, the storage device 140, and / or the positioning and navigation system 160 via the network 150. For another example, the server 120 may be directly connected to the terminal device 130, the detection unit 112, the vehicle 110, and / or the storage device 140 to access the stored information and / or data. In some embodiments, the server 120 may be implemented on a cloud platform or on a vehicle-mounted computer. By way of example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, a multi-cloud, etc., or any combination thereof. In some embodiments, the server 120 may be implemented on a cloud platform or on a vehicle-mounted computer including the one described in this application. Figure 2 One or more of the components shown are implemented on computing device 200 .

[0035] In some embodiments, the server 120 may include a processing device 122. The processing device 122 may process information and / or data related to distance measurement to perform one or more functions described in the present application. For example, the processing device 122 may identify at least one target line in the image based on the object in the image, and determine the distance between the vehicle 110 (or a camera mounted on the vehicle 110) and the object based on the at least one target line. In addition, the processing device 122 may determine the driving path of the vehicle 110 based on the distance between the vehicle 110 and the object. That is, the processing device 122 may be configured as a planning component of the vehicle 110. In some embodiments, the processing device 122 may include one or more processing engines (e.g., a single-core processing engine or a multi-core processor). By way of example only, the processing device 122 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), an application specific instruction set processor (ASIP), an image processing unit (GPU), a physical operation processing unit (PPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a controller, a microcontroller unit, a reduced instruction set computer (RISC), a microprocessor, etc., or any combination thereof. In some embodiments, the processing device 122 may be integrated into the vehicle 110 and / or the terminal device 130 .

[0036] In some embodiments, the terminal device 130 may include a mobile device 130-1, a tablet computer 130-2, a laptop computer 130-3, a built-in device in a vehicle 130-4, a wearable device 130-5, etc., or any combination thereof. In some embodiments, the mobile device 130-1 may include a smart home device, a smart mobile device, a virtual reality device, an augmented reality device, etc., or any combination thereof. Smart home devices may include smart lighting devices, smart appliance control devices, smart monitoring devices, smart TVs, smart cameras and intercoms, etc., or any combination thereof. Smart mobile devices may include smart phones, personal digital assistants (PDAs), gaming devices, navigation devices, point-of-sale (POS) devices, etc., or any combination thereof. Virtual reality or augmented reality devices may include virtual reality helmets, virtual reality glasses, virtual reality goggles, augmented reality helmets, augmented reality glasses and augmented reality goggles, etc., or any combination thereof. For example, virtual reality devices and / or augmented reality devices may include Google TMGlass, Oculus Rift, HoloLens, Gear VR, etc. In some embodiments, the vehicle-mounted device 130-4 may include a vehicle-mounted computer, a vehicle-mounted TV, etc. In some embodiments, the wearable device 130-5 may include a smart bracelet, smart shoes and socks, smart glasses, a smart helmet, a smart watch, smart clothing, a smart backpack, a smart accessory, etc., or any combination thereof. In some embodiments, the terminal device 130 may be a device with a positioning technology for locating the position of the terminal device 130. In some embodiments, the server 120 may be integrated into the vehicle 110 and / or the terminal device 130.

[0037] The storage device 140 may store data and / or instructions. In some embodiments, the storage device 140 may store data obtained from the vehicle 110, the detection unit 112, the processing device 122, the terminal device 130, the positioning and navigation system 160, and / or an external storage device. For example, the storage device 140 may store information and / or data associated with distance measurement (e.g., at least two preset reference lines, at least one target line, at least one measurement point). In some embodiments, the storage device 140 may store data and / or instructions used by the server 120 to execute or complete the exemplary method described in this application. For example, the storage device 140 may store instructions that the processing device 122 may execute or use to determine the distance between the vehicle 110 (or a camera mounted on the vehicle 110) and the object. In some embodiments, the storage device 140 may include a mass storage, a removable storage, a volatile read-write memory, a read-only memory (ROM), etc., or any combination thereof. Exemplary mass storage may include a disk, an optical disk, a solid-state drive, etc. Exemplary removable storage devices may include a flash drive, a floppy disk, an optical disk, a memory card, a zip disk, a tape, etc. Exemplary volatile read-write memory may include random access memory (RAM). Exemplary RAM may include dynamic random access memory (DRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), static random access memory (SRAM), thyristor random access memory (T-RAM) and zero capacitance random access memory (Z-RAM), etc. Exemplary read-only memory may include mask read-only memory (MROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), compact disk read-only memory (CD-ROM) and digital versatile disk read-only memory, etc. In some embodiments, the storage device 140 may be implemented on a cloud platform. By way of example only, a cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, a multi-cloud, etc., or any combination thereof.

[0038] In some embodiments, the storage device 140 can be connected to the network 150 to communicate with one or more components of the autonomous driving system 100 (e.g., the server 120, the terminal device 130, the detection unit 112, the vehicle 110, and / or the positioning). One or more components of the autonomous driving system 100 can access the data or instructions stored in the storage device 140 via the network 150. In some embodiments, the storage device 140 can be directly connected to or communicate with one or more components of the autonomous driving system 100 (e.g., the server 120, the terminal device 130, the detection unit 112, the vehicle 110, and / or the positioning navigation system 160). In some embodiments, the storage device 140 can be part of the server 120. In some embodiments, the storage device 140 can be integrated into the vehicle 110.

[0039] The network 150 may facilitate the exchange of information and / or data. In some embodiments, one or more components of the autonomous driving system 100 (e.g., the server 120, the terminal device 130, the detection unit 112, the vehicle 110, the storage device 140, or the positioning and navigation system 160) may send information and / or data to other components of the autonomous driving system 100 via the network 150. For example, the server 120 may obtain information and / or data associated with distance measurement (e.g., images, at least two preset reference lines, at least one target line) from the vehicle 110, the terminal device 130, the storage device 140, and / or the positioning and navigation system 160 via the network 150. In some embodiments, the network 150 may be a wired network or a wireless network, etc., or any combination thereof. As examples only, the network 150 may include a cable network, a wired network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, a near field communication (NFC) network, etc. or any combination thereof. In some embodiments, the network 150 may include one or more network access points. For example, the network 150 may include a wired or wireless network access point (e.g., 150-1, 150-2), through which one or more components of the autonomous driving system 100 may connect to the network 150 to exchange data and / or information.

[0040] The positioning and navigation system 160 can determine information associated with an object (e.g., a terminal device 130, a vehicle 110, etc.). In some embodiments, the positioning and navigation system 160 may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a compass navigation system (COMPASS), a Beidou navigation satellite system, a Galileo positioning system, a quasi-zenith satellite system (QZSS), etc. The information may include the location, altitude, speed or acceleration, current time, etc. of the object. The positioning and navigation system 160 may include one or more satellites, such as satellite 160-1, satellite 160-2, and satellite 160-3. Satellites 160-1 to 160-3 may determine the above information independently or jointly. The positioning and navigation system 160 may send the above information to the server 120, the vehicle 110, and / or the terminal device 130 via a wireless connection.

[0041] It should be understood by those skilled in the art that when an element (or component) of the autonomous driving system 100 is executed, the element may be executed by an electrical signal and / or an electromagnetic signal. For example, when the terminal device 130 sends a request to the server 120, the processor of the terminal device 130 may generate an electrical signal encoding the request. The processor of the terminal device 130 may then transmit the electrical signal to an output port. If the terminal device 130 communicates with the server 120 via a wired network, the output port may be physically connected to a cable, which may also transmit the electrical signal to an input port of the server 120. If the terminal device 130 communicates with the server 120 via a wireless network, the output port of the terminal device 130 may be one or more antennas that convert the electrical signal into an electromagnetic signal. In an electronic device such as the terminal device 130 and / or the server 120, when its processor processes an instruction, issues an instruction, and / or performs an action, the instruction and / or action is performed by an electrical signal. For example, when the processor retrieves or saves data from a storage medium (e.g., storage device 140), it can send an electrical signal to a read / write device of the storage medium, which can read or write structured data in the storage medium. The structured data can be transmitted to the processor via a bus of the electronic device in the form of an electrical signal. Here, the electrical signal can refer to one electrical signal, a series of electrical signals, and / or at least two discontinuous electrical signals.

[0042] It should be noted that the above statements are for exemplary purposes only and are not intended to limit the scope of the present application. For those skilled in the art, various changes and modifications can be made under the guidance of the present application. However, these changes and modifications will not deviate from the scope of the present application. For example, in addition to autonomous driving, the present application can also be applied to various distance measurement scenarios, such as road mapping, road maintenance, etc.

[0043] Figure 22 is a schematic diagram of exemplary hardware and / or software components of an exemplary computing device according to some embodiments of the present application. In some embodiments, the server 120 and / or the terminal device 130 can be implemented on the computing device 200. For example, the processing device 122 can be implemented on the computing device 200 and configured to perform the functions of the processing device 122 disclosed in the present application.

[0044] The computing device 200 can be used to implement any component of the autonomous driving system 100 of the present application. For example, the processing device 122 of the autonomous driving system 100 can be implemented on the computing device 200 through its hardware, software program, firmware or a combination thereof. Although only one such computer is shown for convenience, the computer functions related to the autonomous driving system 100 described herein can be implemented in a distributed manner on multiple similar platforms to distribute the processing load.

[0045] The computing device 200 may include a connected communication port (COM) 250 and may be connected to a network (e.g., network 150) to facilitate data communication. The computing device 200 may also include a processor (e.g., processor 220) for executing program instructions, which may exist in the form of one or more processors (e.g., logic circuits). For example, the processor may include an interface circuit and a processing circuit therein. The interface circuit may be configured to receive an electrical signal from the bus 210, wherein the electrical signal encodes structured data and / or instructions for the processing circuit. The processing circuit may perform logical calculations and then determine the conclusion, result, and / or instruction encoding as an electrical signal. The interface circuit may then send an electrical signal from the processing circuit via the bus 210.

[0046] The computing device 200 may further include program storage and different forms of data storage, such as a disk 270, a read-only memory (ROM) 230, or a random access memory (RAM) 240, for storing various data files processed and / or transmitted by the computing device 200. The computing device 200 may also include program instructions stored in the ROM 230, the RAM 240, and / or another type of non-transitory storage medium executed by the processor 220. The methods and / or processes of the present application may be implemented as program instructions. The computing device 200 also includes an I / O component 260, which supports input / output between the computing device 200 and other components therein. The computing device 200 may also receive programming and data via network communications.

[0047] For illustration only, only one processor is described in the computing device 200. However, it should be noted that the computing device 200 in the present application may also include multiple processors, and therefore, the operations performed by one processor as described in the present application may also be performed jointly or separately by multiple processors. For example, the processor of the computing device 200 performs operations A and B simultaneously. For another example, operations A and B may also be performed jointly or separately by two different processors in the computing device 200 (for example, the first processor performs operation A, the second processor performs operation B, or the first and second processors perform operations A and B jointly).

[0048] Figure 3 1 is a schematic diagram of exemplary hardware and / or software components of an exemplary mobile device according to some embodiments of the present application. In some embodiments, the terminal device 130 can be implemented on the mobile device 300. Figure 3 As shown, mobile device 300 may include a communication platform 310, a display 320, a graphics processing unit (GPU) 330, a central processing unit (CPU) 340, an I / O 350, a memory 360, a mobile operating system (OS) 370, and a storage 390. In some embodiments, any other suitable components, including but not limited to a system bus or a controller (not shown), may also be included in mobile device 300.

[0049] In some embodiments, the mobile operating system 370 (e.g., iOS TM 、Android TM 、Windows Phone TM ) and one or more applications 380 may be loaded from storage 390 into memory 360 for execution by CPU 340. Application 380 may include a browser or any other suitable mobile application for receiving and presenting location-related or other information from processing device 122. User interaction with the information stream may be enabled through I / O 350 and provided to processing device 122 and / or other components of autonomous driving system 100 through network 150.

[0050] In order to implement the various modules, units and functions described in this application, a computer hardware platform can be used as a hardware platform for one or more components described herein. A computer with user interface elements can be used to implement a personal computer (PC) or any other type of workstation or terminal device. If the computer is properly programmed, the computer can also be used as a server.

[0051] Figure 4 is a block diagram of an exemplary processing device according to some embodiments of the present application. Figure 4As shown, the processing device 122 may include an acquisition module 402 , a target line identification module 404 , and a distance determination module 406 .

[0052] The acquisition module 402 may be configured to acquire an image captured by a camera (e.g., a monocular camera). In some embodiments, the image may include a two-dimensional (2D) image, a three-dimensional (3D) image, a three-dimensional (4D) image, etc., or any combination thereof. The image may include objects, such as vehicles (e.g., bicycles, motors, cars, buses, trucks), pedestrians, animals (e.g., dogs, cats), obstacles (e.g., stones, roadblocks, wooden boards, telephone poles, etc.).

[0053] The target line recognition module 404 may be configured to recognize at least one target line in the image based on the object. In some embodiments, the at least one target line may be associated with at least two preset reference lines, wherein the at least two preset reference lines correspond to their horizontal reference distances to the camera (also referred to as "horizontal reference distances to the vehicle 110"), respectively. In some embodiments, the target line recognition module 404 may project at least one of the at least two preset reference lines onto the image, and determine at least one predicted line corresponding to at least one of the at least two preset reference lines. In addition, the target line recognition module 404 may recognize at least one target line based on at least one predicted line and the object. For example, the target line recognition module 404 may determine at least one measurement point associated with the object in the image, and recognize at least one target line in the image based on the at least one measurement point and the at least one predicted line. In some embodiments, the target line recognition module 404 may include a measurement point determination unit (not shown) configured to determine at least one measurement point.

[0054] The distance determination module 406 can be configured to determine the distance between the camera and the object based on at least one target line. In some embodiments, for each of the at least one target line, the distance determination module 406 can determine a target distance corresponding to the target line based on the height and horizontal distance corresponding to the target line. The distance determination module 406 can then determine the distance between the camera and the object based on at least one target distance corresponding to the at least one target line. Further description of distance determination can be found elsewhere in this application (e.g., Fig.11 and its description).

[0055] The modules in the processing device 122 can be configured to communicate with each other via a wired connection or a wireless connection. The wired connection may include a metal cable, an optical cable, a hybrid cable, etc., or any combination thereof. The wireless connection may include a local area network (LAN), a wide area network (WAN), Bluetooth, Zigbee, a near field communication (NFC), etc., or any combination thereof. Two or more modules may be combined into a single module, and any one module may be divided into two or more units. For example, the processing device 122 may include a storage module (not shown) for storing information and / or data (e.g., at least two preset reference lines, at least one prediction line, at least one target line, at least one measurement value) associated with the distance measurement.

[0056] Figure 5 5 is a flowchart of an exemplary process for determining the distance between a camera and an object according to some embodiments of the present application. In some embodiments, process 500 may be implemented as a set of instructions (e.g., an application) stored in ROM 230 or RAM 240. Processor 220 and / or Figure 4 The modules in the embodiment may execute the set of instructions, and when executing the instructions, the processor 220 and / or the modules may be configured to perform the process 500. The operations of the exemplary process described below are intended to be illustrative. In some embodiments, the process 500 may be completed by one or more additional operations not described and / or not by one or more operations discussed in this application. In addition, this application is not intended to limit Figure 5 and the order of operations of the following processes.

[0057] In step 501 , the processing device 122 (eg, the acquisition module 402 ) (eg, the interface circuit of the processor 220 ) may acquire an image captured by a camera (eg, a monocular camera).

[0058] Combination Figure 1 As described, the camera may be mounted on the vehicle 110 and configured to capture images associated with the surrounding environment of the vehicle 110. In some embodiments, the images may include two-dimensional (2D) images, three-dimensional (3D) images, four-dimensional (4D) images, etc., or any combination thereof. In some embodiments, the images may include objects. The objects may include vehicles (e.g., bicycles, motors, cars, buses, trucks), pedestrians, animals (e.g., dogs, cats), obstacles (e.g., stones, roadblocks, wooden boards, poles), etc.

[0059] At step 503 , the processing device 122 (eg, the target line identification module 404 ) (eg, the processing circuit of the processor 220 ) may identify at least one target line in the image based on the object.

[0060] In some embodiments, at least one target line may be associated with at least two preset reference lines, wherein each of the at least two preset reference lines corresponds to a horizontal reference distance from the camera (which may also be considered as a horizontal reference distance from the vehicle 110). The horizontal reference distance may be a default setting of the autonomous driving system 100, or may be adjustable in different situations. In some embodiments, the processing device 122 may determine at least one target line based on at least one predicted line corresponding to at least one of the at least two preset reference lines. More description of preset reference lines may be found elsewhere in this application (e.g., Figures 6A-6C and its description).

[0061] In some embodiments, the processing device 122 may determine at least one measurement point associated with an object in the image, and identify at least one target line in the image based on the at least one measurement point and the at least one predicted line. For example, the processing device 122 may determine at least one predicted line on or between which at least one measurement point is located as at least one target line. For another example, the processing device 122 may determine at least one predicted line near at least one measurement point as at least one target line. As used herein, "near..." means that the distance between any one of the at least one measurement point and the at least one target line is less than a distance threshold. For another example, the processing device 122 may select at least one candidate line from at least one predicted line based on the object (e.g., at least one measurement point), and determine at least one target line by performing an interpolation operation on the at least one candidate line. As another example, the processing device 122 may determine at least two predicted lines corresponding to at least two preset reference lines as at least one target line. More instructions for determining at least one target line may be found elsewhere in this application (e.g., Figures 8A-8F and its description).

[0062] In some embodiments, to determine at least one measurement point, the processing device 122 may use an object detection algorithm to determine a bounding box containing the object, and determine at least one measurement point based on the bounding box. The object detection algorithm may include a region with convolutional neural network features (R-CNN) algorithm, a fast R-CNN algorithm, a faster R-CNN algorithm, a look-only-once (YOLO) algorithm, a single shot multi-box detector (SSD) algorithm, etc., or any combination thereof.

[0063] In some embodiments, the bounding box may be a rectangular box, a box along the 2D outline of the object, a circular box, or any irregular box. In some embodiments, the processing device 122 may select a point above the bounding box as at least one measurement point. For example, the processing device 122 may select the midpoint of the edge of the bounding box as at least one measurement point. For another example, the processing device 122 may select at least one endpoint of the edge of the bounding box as at least one measurement point. In some embodiments, the processing device 122 may select one or more points within the bounding box as at least one measurement point. For example, the processing device 122 may select the center point of the bounding box as at least one measurement point. For another example, the processing device 122 may select multiple points within the bounding box and determine a weighted average position based on the positions of the multiple points as at least one measurement point. As used herein, a weighting coefficient corresponding to a point may be associated with the position of the point. For example, a weighting coefficient corresponding to a point on an object may be greater than a weighting coefficient corresponding to a point outside the object. For another example, a weighting coefficient corresponding to a point closer to the center point of the bounding box may be greater than a weighting coefficient corresponding to a point far from the center point. More descriptions of determining at least one measurement point may be found elsewhere in this application (e.g., Figures 7A-7E and its description).

[0064] At step 505 , the processing device 122 (eg, the distance determination module 408 ) (eg, the processing circuitry of the processor 220 ) may determine a distance between the camera and the object based on at least one target line.

[0065] In some embodiments, for each of the at least one target line, the processing device 122 may determine a target distance corresponding to the target line based on a height and a horizontal distance from the camera corresponding to the target line. In addition, the processing device 122 may determine a distance between the camera and the object based on at least one target distance corresponding to the at least one target line.

[0066] In some embodiments, processing device 122 may determine the distance between the camera and the object based on an average or weighted average of at least one target distance. In some embodiments, processing device 122 may determine the distance between the camera and the object based on a sum or weighted sum of at least one target distance. Further description of distance determination may be found elsewhere in this application (e.g., Fig.11 and its description).

[0067] In some embodiments, the processing device 122 may control the camera to continuously capture images and determine whether an object (e.g., a pedestrian, an obstacle) is detected in the image. In response to determining that the object is detected, the processing device 122 may determine the distance between the camera and the object. In some embodiments, the processing device 122 may control the camera to capture images associated with an object (e.g., a moving vehicle) in real time and determine the distance between the camera and the object in real time.

[0068] It should be noted that the above description of the process 500 is only for example and illustration, and does not limit the scope of application of the present application. For those skilled in the art, various changes and modifications can be made under the teachings of the present application. However, these changes and modifications do not deviate from the scope of the present application. For example, operation 503 may include sub-operations, wherein the processing device 122 can determine at least one measurement point associated with the object. For another example, one or more other optional operations (e.g., storage operations) can be added at other locations in the process 500. In the storage operation, the processing device 122 can store information and / or data (e.g., at least two preset reference lines, at least one target line, at least one measurement point) associated with the distance measurement in a storage device (e.g., device 140) disclosed elsewhere in the present application.

[0069] Fig. 6A and Figure 6B is a schematic diagram of the positional relationship between a camera and at least two preset reference lines according to some embodiments of the present application. As shown in the figure, Fig. 6A is a side view of the camera and at least two preset reference lines, Figure 6B A top view of the camera and at least two preset reference lines.

[0070] like Fig. 6A and 6B As shown, the camera 604 can be mounted on the top of the vehicle 110, and at least two preset reference lines 606 can be perpendicular to the optical axis 608 of the camera 604. Each of the at least two preset reference lines 606 can correspond to a corresponding horizontal reference distance (e.g., L1, L2) to the camera 604. In some embodiments, the at least two preset reference lines 606 can be equidistant. That is, the at least two preset reference lines 606 can be lines with equidistant intervals along the moving direction of the vehicle 110. The equidistant intervals can be the default setting of the autonomous driving system 100, or can be adjusted in different situations. For example, the equidistant intervals can be 10 cm, 50 cm, 1 m, 2 m, 5 m, etc. In some embodiments, the at least two preset reference lines 606 may be non-equidistant. That is, the at least two preset reference lines can be lines with non-equidistant intervals along the moving direction of the vehicle 110.

[0071] In some embodiments, at least two preset reference lines 606 may be viewed as at least two virtual lines on a road as the camera 604 (or vehicle 110) moves. In some cases, since the road may be uneven (i.e., the heights (e.g., altitudes) of different locations on the road may be different), the heights of the preset reference lines may be different as the vehicle 110 travels along the road. In some embodiments, the heights of the preset reference lines may be obtained from map information stored in a storage device (e.g., storage device 140) disclosed elsewhere in this application.

[0072] Figure 6C FIG. 1 is a schematic diagram of an exemplary prediction line corresponding to at least two preset reference lines according to some embodiments of the present application. Figure 6C As shown, one or more prediction lines 656 can be generated by projecting at least one of the at least two preset reference lines 606 onto the imaging plane 652 of the camera 604. Taking a specific preset reference line as an example, the specific preset reference line corresponds to a specific horizontal reference distance to the camera, and therefore, the prediction line corresponding to the specific preset reference line corresponds to the specific horizontal reference distance. In combination with step 503, the processing device 122 can identify at least one target line based on the one or more prediction lines 656. Therefore, in combination with step 505, the target distance corresponding to each of the at least one target line can be determined based on the horizontal reference distance corresponding to the prediction line 656 (or the preset reference line 606) and the height of the preset reference line 606.

[0073] Figures 7A-7E FIG. 1 is a schematic diagram of exemplary measurement points associated with an object according to some embodiments of the present application. Figures 7A-7E 702 in the figure refers to an image including an object (e.g., a vehicle), and 704 refers to a bounding box including the object. For the purpose of illustration, the present application takes a rectangular bounding box as an example, but is not limited thereto. As described in other parts of the present application, the bounding box can be a circular box, a box along the 2D outline of the object, or any regular or irregular box.

[0074] In conjunction with step 503, the processing device 122 may select one or more points on or within the bounding box 704 as at least one measurement point. Fig. 7A As shown, the processing device 122 may select the midpoint 706 of the edge of the bounding box 704 as at least one measurement point. Figure 7B As shown, the processing device 122 may select two endpoints 716 of the edge of the bounding box 704 as at least one measurement point. Figure 7C As shown, the processing device 122 may select the center point 726 of the bounding box 704 as at least one measurement point. Fig.7DAs shown, the processing device 122 may select multiple random points 736 within the bounding box 704 as at least one measurement point. Alternatively or additionally, the processing device 122 may determine a weighted average position of the positions of the multiple random points 736 as at least one measurement point. As used herein, the weighting coefficient corresponding to a point on the object may be greater than the weighting coefficient corresponding to a point outside the object. Fig. 7E As shown, the processing device 122 may select a midpoint 746 - 1 of an edge of the bounding box 704 and a center point 746 - 2 of the bounding box 704 as at least one measurement point.

[0075] It should be noted that the above description is for illustrative purposes only and is not intended to limit the scope of the present application. For those skilled in the art, various changes and modifications may be made under the teachings of the present application. However, these changes and modifications do not deviate from the scope of the present application. For example, the bounding box may be a circular box, and the processing device 122 may select one or more points on the circumference of the circular box and / or in the circular box as at least one measurement point. For another example, the bounding box may be a box along the 2D contour of the object, and the processing device 122 may select one or more points on and / or in the box as at least one measurement point. For another example, the processing device 122 may randomly select a point on the bounding box as at least one measurement point.

[0076] Figures 8A-8F is a schematic diagram of an exemplary process for determining at least one target line based on at least one measurement point according to some embodiments of the present application. Figures 8A-8F As shown, the dotted line represents a predicted line corresponding to a preset reference line, and 912, 962-1, and 962-2 refer to measurement points associated with an object (e.g., a vehicle). For the purpose of illustration, the present application takes a single measurement point or two measurement points as an example, but it should be noted that the number of at least one measurement point can also be three or more.

[0077] like Fig. 8A As shown, the processing device 122 can determine the predicted line 914 where the measurement point 912 is located as at least one target line. Figure 8B As shown, the processing device 122 may determine two predicted lines 924a and 924b as at least one target line, wherein the measurement point 912 is located between the two predicted lines 924a and 924b. Figure 8C As shown, the processing device 122 may determine two prediction lines 934a and 934b, wherein the measurement point 912 is located between the two prediction lines 934a and 934b, determine an interpolated prediction line 934 by performing an interpolation operation on the two prediction lines 934a and 934b, and determine the interpolated prediction line 934 as at least one target line. In this case, the interpolated prediction line corresponds to the interpolated reference line. Fig.8DAs shown, the processing device 122 may determine one or more predicted lines (eg, predicted lines 944a-944e) near the measurement point 912 as at least one target line. Fig. 8E As shown, the processing device 122 may determine one or more predicted lines corresponding to a preset reference line between the object (eg, the measurement point 912) and the camera as at least one target line. Fig.8F As shown, the processing device 122 may determine one or more prediction lines (eg, prediction lines 964a - 964e ) between the prediction line 964a where the measurement point 962 - 1 is located and the prediction line 964e where the measurement point 962 - 2 is located as at least one target line.

[0078] It should be noted that the above description is for illustrative purposes only and is not intended to limit the scope of the present application. For those skilled in the art, various changes and modifications may be made under the teachings of the present application. However, these changes and modifications do not deviate from the scope of the present application. For example, the processing device 122 may determine a measurement line connecting at least two measurement points, and determine one or more prediction lines intersecting the measurement line as at least one target line.

[0079] Fig. 9 900 is a flowchart of an exemplary process for identifying at least one target line according to some embodiments of the present application. In some embodiments, process 900 may be implemented as a set of instructions (e.g., an application program) stored in memory ROM 230 or RAM 240. Processor 220 and / or Figure 4 The module in the can execute the set of instructions. When executing the instructions, the processor 220 and / or the module can be configured to perform the process 900. The operation of the following exemplary process is intended to be illustrative. In some embodiments, the process 900 can be completed by one or more additional operations not described and / or not by one or more operations discussed in this application. In addition, this application does not limit Fig. 9 and the following describes the order of operations of the process.

[0080] Step 901, for at least one of the at least two preset reference lines, the processing device 122 (e.g., the target line identification module 404) (e.g., the interface circuit of the processor 220) may obtain the height of the preset reference line based on the map information. As used herein, the "height" of the preset reference line may refer to the height of a portion of the road where the preset reference line is located (e.g., the portion where the center point of the preset reference line is located).

[0081] In some embodiments, the processing device 122 may obtain the height of the preset reference line based on the map information related to the road. Taking "autonomous driving" as an example, the processing device 122 may obtain the height of the preset reference line from a high-definition map containing centimeter-level or millimeter-level precision information. In some embodiments, the high-definition map may be generated online or offline. For example, the high-definition map may be generated based on at least two detection units (e.g., Figure 1 A high-definition map is generated offline based on data (e.g., point cloud data) captured by the detection units described in the test vehicle for measuring the trip. When the test vehicle moves along the road, at least two detection units can generate point cloud data associated with the surrounding environment of the test vehicle. In addition, a processing device (e.g., processing device 122) can generate at least two high-definition maps corresponding to different geographic areas based on the point cloud data, and store the at least two high-definition maps in a storage device (e.g., storage device 140 of the autonomous driving system 100). Therefore, the processing device 122 can access the storage device and obtain the corresponding high-definition map based on the estimated position of the road portion where the preset reference line is located.

[0082] Step 903 , for at least one of the preset reference lines, the processing device 122 (eg, the target line identification module 404 ) (eg, the processing circuit of the processor 220 ) may select at least two reference points on the preset reference line.

[0083] In some embodiments, the processing device 122 may select two endpoints of a preset reference line as at least two reference points. In some embodiments, the processing device 122 may randomly select two or more points (e.g., Fig.10 Point A and point B) are shown as at least two reference points.

[0084] In step 905, the processing device 122 (e.g., the target line identification module 404) (e.g., the processing circuit of the processor 220) may project at least two reference points onto the image (or imaging plane) based on the horizontal reference distance to the camera and the height of the preset reference line. In some embodiments, the processing device 122 may project at least two reference points onto the image (or imaging plane) by performing a perspective projection operation.

[0085] by Fig.10 Taking point A as an example, the processing device 122 can determine the projection point A' in the imaging plane according to formula (1):

[0086]

[0087] Among them, x a ,y a and z aThey represent the x-axis coordinate, y-axis coordinate, and z-axis coordinate of point A in the camera coordinate system, respectively. 0a and 0a The x-axis and y-axis coordinates of the projection point A' in the image coordinate system are represented by wa ,y wa and z wa They represent the x-axis coordinate, y-axis coordinate, and z-axis coordinate of point A in the world coordinate system, respectively. f represents the focal length of the camera. represents the perspective projection matrix, represents the extrinsic parameter matrix associated with the camera, where R is the translation transformation matrix and T is the rotation transformation matrix.

[0088] In some embodiments, taking point A as an example, the processing device 122 may determine the coordinates of point A in the world coordinate system (i.e., x wa ,y wa and z wa For example, the x-axis coordinate (i.e., x wa ) can be the height difference between the preset reference line and the camera, and the z-axis coordinate (i.e., z wa ) can be the horizontal reference distance between the preset reference line and the camera, and the y-axis coordinate (i.e., y wa ) may be the position coordinates of the point along the preset reference line. In addition, the processing device 122 may convert the coordinates of point A in the world coordinate system (i.e., x wa ,y wa and z wa ) determines the coordinates of point A in the camera coordinate system (i.e., x a ,y a and z a ):

[0089]

[0090] Wherein R represents the translation transformation matrix of the translation parameters (for example, a 3*3 matrix), T represents the rotation transformation matrix of the rotation parameters (for example, a 3*1 matrix), wherein the translation parameters and the rotation parameters can be collectively referred to as external parameters related to the camera.

[0091] In step 907, the processing device 122 (e.g., the target line identification module 404) (e.g., the processing circuit of the processor 220) may determine a predicted line corresponding to a preset reference line in the image based on the at least two projected reference points. In some embodiments, the processing device 122 may determine a line (e.g., Fig.10 The line connecting point A′ and point B′ shown in FIG. 1 is a prediction line.

[0092] At step 909 , the processing device 122 (eg, the target line identification module 404 ) (eg, the processing circuitry of the processor 220 ) may identify at least one target line based on at least one predicted line corresponding to at least one of the at least two preset reference lines and the object.

[0093] In conjunction with step 503, the processing device 122 may determine at least one measurement point associated with the object in the image, and identify at least one target line in the image based on the at least one measurement point. For example, the processing device 122 may determine at least one predicted line on or between which the at least one measurement point is located as the at least one target line. For another example, the processing device 122 may select at least one candidate line from the at least one predicted line based on the object, and determine the at least one target line by performing an interpolation operation on the at least one candidate line. More descriptions about determining at least one target line may be found elsewhere in this application (e.g., Figures 8A-8F and its description).

[0094] It should be noted that the above description of process 900 is only for example and explanation, and does not limit the scope of application of the present application. For those skilled in the art, various changes and modifications can be made under the teachings of the present application. However, these changes and modifications do not deviate from the scope of the present application. For example, one or more other optional operations (e.g., storage operations) can be added elsewhere in process 900. In the storage operation, the processing device 122 can store information and / or data associated with the distance measurement (e.g., at least two preset reference lines, at least one prediction line, at least one target line) in a storage device (e.g., storage device 140) disclosed elsewhere in the present application. For another example, the processing device 122 can project the preset reference line directly onto the image (or imaging plane) based on the horizontal reference distance to the camera and the height of the preset reference line, and determine the prediction line corresponding to the preset reference line in the image.

[0095] Fig.10 FIG. 1 is a schematic diagram of an exemplary process for projecting a preset reference line onto an image according to some embodiments of the present application. As shown in the figure, o w _x w y w z w represents the world coordinate system, o_xyz represents the camera coordinate system, and o0_x0y0 represents the image coordinate system.

[0096] For the world coordinate system, the origin (i.e., o w ) can be any point in the environment (e.g., the optical center of the camera, the physical center of the camera, the center of the roof of the vehicle 110), and the three axes (i.e., the x-axis, the y-axis, and the z-axis) can be any direction perpendicular to each other. w) can be the physical center point of the camera, the x-axis (i.e., o w x w ) can be perpendicular to the horizontal plane, the z-axis (i.e., o w z w ) can be parallel to the horizontal plane and the optical axis of the camera, and the y-axis (i.e., o w y w ) can be parallel to the horizontal plane and perpendicular to o w z w As shown in the figure, the preset reference line is perpendicular to the z-axis (i.e., w z w ) (e.g., the optical axis of the camera) and parallel to the y-axis (i.e., o w y w ). Therefore, the coordinates of any point on the preset reference line in the world coordinate system can be expressed as (x wi ,y wi 、z wi ), where x wi The y may be determined based on the height of a preset reference line (eg, the height difference between the preset reference line and the camera). wi It can be determined based on the position of the point on the preset reference line, z wi It can be the horizontal reference distance from the preset reference line to the camera.

[0097] For the camera coordinate system, the origin (i.e., o) may be the optical center of the camera, the z-axis (i.e., oz) may be in the direction along the optical axis of the camera, and the x-axis (i.e., ox-axis) and the y-axis (i.e., oy-axis) may be parallel to the imaging plane 1006. As described in conjunction with operation 905, the coordinates of a point in the world coordinate system may be transformed into corresponding coordinates in the camera coordinate system according to formula (2).

[0098] For the image coordinate system along the imaging plane 1006, the origin (i.e., o0) may be the intersection of the optical axis of the camera and the imaging plane 1006, the x-axis (i.e., o0x0) may be parallel to the x-axis of the camera coordinate system, and the y-axis (i.e., o0y0) may be parallel to the y-axis of the camera coordinate system. In combination with step 905, the processing device 122 may select at least two reference points (e.g., point A, point B) on the preset reference line 1002 according to formula (1), and project the at least two reference points onto the imaging plane 1006. Alternatively or additionally, the processing device 122 may directly project the preset reference line 1002 onto the imaging plane 1006 according to formula (1).

[0099] Fig.111 is a flowchart of an exemplary process for determining the distance between a camera and an object according to some embodiments of the present application. In some embodiments, the process 1100 may be implemented as a set of instructions (e.g., an application program) stored in the memory ROM 230 or RAM 240. The processor 220 and / or Figure 4 The module in the can execute the set of instructions. When executing the instructions, the processor 220 and / or the module can be configured to perform the process 1100. The operation of the following exemplary process is intended to be illustrative. In some embodiments, the process 1100 can be completed by one or more additional operations not described and / or not by one or more operations discussed in this application. In addition, this application does not limit Fig.11 and the order of operations of the following processes.

[0100] Step 1101 , for each of at least one target line, the processing device 122 (eg, the distance determination module 406 ) (eg, the processing circuit of the processor 220 ) may obtain a height corresponding to the target line based on map information.

[0101] Combined with step 503 and the accompanying drawings, refer to Figures 8A-8F , at least one target line may be determined based on at least one of the at least two preset reference lines (e.g., based on at least one predicted line corresponding to at least one of the at least two preset reference lines). Therefore, for each of the at least one target line, the processing device 122 may determine the height corresponding to the target line based on the height of at least one of the at least two preset reference lines. For example, assuming that the target line is a predicted line, the processing device 122 may obtain the height of the preset reference line corresponding to the predicted line. For another example, assuming that the target line is an interpolated predicted line determined based on two predicted lines (e.g., Figure 8C line 934 in the figure), the processing device 122 can obtain the heights of two preset reference lines corresponding to the two predicted lines, and determine the height corresponding to the target line (i.e., the interpolated predicted line) (which can be regarded as corresponding to the height of the interpolated reference line) by performing an interpolation operation on the two heights.

[0102] In conjunction with step 901, the “height” of the preset reference line may refer to the height of a portion of the road where the preset reference line is located. The processing device 122 may access a storage device (e.g., the storage device 140), and retrieve a corresponding high-definition map based on the estimated position of the portion of the road where the preset reference line is located to obtain the height of the preset reference line.

[0103] In step 1103 , the processing device 122 (eg, the distance determination module 406 ) (eg, the processing circuit of the processor 220 ) may obtain a horizontal distance from the camera corresponding to the target line.

[0104] As described above, for each of the at least one target line, the processing device 122 may determine the horizontal distance from the camera corresponding to the target line based on the horizontal reference distance from at least one of the at least two preset reference lines to the camera. For example, assuming that the target line is a predicted line, the processing device 122 may obtain the horizontal reference distance of the preset reference line corresponding to the predicted line. For another example, assuming that the target line is an interpolated predicted line determined based on two predicted lines (e.g., Figure 8C 934 in the figure), the processing device 122 can obtain the horizontal reference distances of the two preset reference lines corresponding to the two predicted lines, and determine the horizontal reference distance corresponding to the target line (which can be regarded as the horizontal reference distance from the corresponding interpolated reference line to the camera) by performing an interpolation operation on the two horizontal reference distances. In some embodiments, the processing device 122 can obtain the horizontal reference distance from the preset reference line to the camera by searching a storage device (e.g., storage device 140) disclosed elsewhere in this application.

[0105] At step 1105 , the processing device 122 (eg, the distance determination module 406 ) (eg, the processing circuitry of the processor 220 ) may determine a target distance corresponding to the target line based on the height and the horizontal distance.

[0106] In some embodiments, for each of the at least one target line, the processing device 122 may determine the target distance corresponding to the target line according to the following formula (3):

[0107]

[0108] Among them, d i Refers to the i-th target distance corresponding to the i-th target line, m i Refers to the horizontal distance corresponding to the i-th target line, n i Refers to the height corresponding to the i-th target line, and N refers to the height of the camera. As used herein, the "height" of the camera may refer to the height of a portion of the road where the camera (or vehicle 110) is located (e.g., the portion where the center point of the camera or vehicle 110 is located). It can be seen that the target distance corresponding to the target line can be regarded as the distance between the camera and a reference line corresponding to the target line (e.g., a preset reference line or an interpolated reference line).

[0109] In some embodiments, the processing device 122 may determine the target distance corresponding to the target line according to the following formula (4):

[0110]

[0111] Among them, D i Refers to the i-th target distance corresponding to the i-th target line, m iRefers to the horizontal distance corresponding to the i-th target line, m i-1 Refers to the horizontal distance corresponding to the i-1th target line, n i refers to the height corresponding to the i-th target line, n i-1 Refers to the height corresponding to the i-1th target line. It can be seen that the target distance corresponding to the target line can be regarded as the distance between the corresponding reference line relative to the camera and the previous reference line (or, the distance between the corresponding reference line and the camera, which reference line is the closest reference line to the camera).

[0112] At step 1107 , the processing device 122 (eg, the distance determination module 406 ) (eg, the processing circuitry of the processor 220 ) may determine the distance between the camera and the object based on at least one target distance corresponding to at least one target line.

[0113] In some embodiments, the processing device 122 may determine the distance between the camera and the object based on at least one target distance according to the following formula (5):

[0114]

[0115] Among them, d T Refers to the distance between the camera and the object, d i refers to the distance of the ith target corresponding to the ith target line, w i Refers to the weighting coefficient corresponding to the i-th target distance, and n refers to the count of at least one target line. As used herein, the shorter the target distance, the larger the weighting coefficient corresponding to the target distance may be.

[0116] In some embodiments, the processing device 122 may determine the distance between the camera and the object based on at least one target distance according to the following formula (6):

[0117]

[0118] Among them, D T Refers to the distance between the camera and the object, D i refers to the distance of the i-th target corresponding to the i-th target line, W i Refers to the weighting coefficient corresponding to the i-th target distance, and n refers to the count of at least one target line. As used herein, the smaller the horizontal distance to the camera corresponding to the target line, the larger the weighting coefficient corresponding to the target distance may be.

[0119] It should be noted that the above description of process 900 is only for example and illustration, and does not limit the scope of application of the present application. For those skilled in the art, various changes and modifications can be made under the guidance of the present application. However, these changes and modifications do not deviate from the scope of the present application.

[0120] Fig. 12A FIG. 1 is a schematic diagram of a side view of an exemplary distance between a camera and an object according to some embodiments of the present application. Fig. 12A As shown, 1210 refers to a camera installed on the vehicle 110, and 1220 refers to a reference line corresponding to the target line (for example, a preset reference line or an interpolated reference line, which may be collectively referred to as a "reference line" for the sake of simplicity). It can be seen that the vehicle 110 can move along the slope AC. Fig.11 As described above, the processing device 122 may obtain a height (e.g., n0) corresponding to the target line (we assume that the height of the camera 1210 is 0) and a horizontal distance to the camera corresponding to the target line (e.g., m0), and determine the target distance (e.g., the hypotenuse d0 of the right triangle ABC) according to formula (3) based on the height and the horizontal distance. In addition, the processing device 122 may determine the distance between the camera 1210 and the object (which may be located on the reference line 1220) (not shown) based on at least one target distance corresponding to at least one target line. Assuming that there is only one target line, the processing device 122 may determine the distance between the camera 1210 and the object as approximately the target distance (e.g., d0).

[0121] Fig. 12B FIG. 1 is a schematic diagram of a side view of an exemplary distance between a camera and an object according to some embodiments of the present application. Fig. 12B As shown, 1242, 1244 and 1246 refer to reference lines corresponding to the target line. It can be seen that the vehicle 110 can travel along an uneven road. Fig.11 As described above, for each target line, the processing device 122 may obtain the height corresponding to the target line and the horizontal distance to the camera corresponding to the target line. For example, for convenience, assuming that the height of the camera 1210 is 0, for the target line corresponding to the reference line 1242, the processing device 122 may obtain the height n1 and the horizontal distance m1; for the target line corresponding to the reference line 1244, the processing device 122 may obtain the height n2 and the horizontal distance m2; for the target line corresponding to the reference line 1246, the processing device 122 may obtain the height n3 and the horizontal distance m3.

[0122] In addition, for each target line, the processing device 122 may determine the target distance corresponding to the target line according to formula (4). For example, for the target line corresponding to the reference line 1242, the processing device 122 may determine the target distance (referred to as the first target distance) according to the following formula:

[0123]

[0124] Wherein, D1 refers to the first target distance, which may be the hypotenuse DF of the triangle ΔDEF.

[0125] For another example, for a target line corresponding to the reference line 1244, the processing device 122 may determine the target distance (referred to as the second target distance) according to the following formula:

[0126]

[0127] Wherein, D2 refers to the second target distance, which may be the hypotenuse FH of the triangle ΔFGH.

[0128] For another example, for a target line corresponding to the reference line 1246, the processing device 122 may determine the target distance (referred to as a third target distance) according to the following formula:

[0129]

[0130] Wherein, D3 refers to the third target distance, which may be the hypotenuse HJ of the triangle ΔHIJ.

[0131] In addition, the processing device 122 may determine the distance between the camera 1210 and the object (which may be located on the reference line 1246) (not shown) based on the target distance corresponding to the target line. For example, the processing device 122 may determine the sum or weighted sum of the target distances as the distance between the camera 1210 and the object according to formula (6).

[0132] It should be noted that the above description is for illustrative purposes only and is not intended to limit the scope of the present application. For those skilled in the art, various changes and modifications may be made under the guidance of the present application. However, these changes and modifications do not deviate from the scope of the present application. For example, the vehicle 110 may move along a curved road, and accordingly, the processing device 122 may divide the curved road into at least two segments and determine at least two distances on the at least two segments according to methods disclosed elsewhere.

[0133] The basic concepts have been described above. Obviously, for those of ordinary skill in the art who have read this application, the above invention disclosure is only for example and does not constitute a limitation of this application. Although not explicitly stated here, those of ordinary skill in the art may make various modifications, improvements and amendments to this application. Such modifications, improvements and amendments are suggested in this application, so such modifications, improvements and amendments still belong to the spirit and scope of the exemplary embodiments of this application.

[0134] At the same time, the present application uses specific words to describe the embodiments of the present application. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of the present application. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, some features, structures or characteristics in one or more embodiments of the present application can be appropriately combined.

[0135] In addition, it will be appreciated by those skilled in the art that various aspects of the present application may be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product, or combination of substances, or any new and useful improvements thereto. Accordingly, various aspects of the present application may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may all be referred to as "blocks," "modules," "units," "components," "engines," or "systems." In addition, various aspects of the present application may take the form of a computer program product embodied in one or more non-temporary computer-readable media having computer-readable program code contained thereon.

[0136] A computer readable signal medium may include a propagated data signal containing computer readable program code, such as in baseband or as part of a carrier wave. Such propagated signals may be in a variety of forms, including electromagnetic, optical, etc., or any suitable combination. A computer readable signal medium may be any computer readable medium other than a computer readable storage medium, which may be connected to an instruction execution system, device or apparatus to communicate, propagate or transmit a program for use. The program code on the computer readable signal medium may be propagated via any suitable medium, including radio, cable, fiber optic cable, RF, etc., or any combination of the above.

[0137] The computer program code required for the operation of each part of the present application can be written in any one or more programming languages, including subject-oriented programming languages ​​(such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, etc.), conventional procedural programming languages ​​(such as C language, Visual Basic, Fortran 1703, Perl, COBOL 1702, PHP, ABAP), dynamic programming languages ​​(such as Python, Ruby and Groovy) or other programming languages. The program code can be run entirely on the user's computer, or run on the user's computer as an independent software package, or run partially on the user's computer and partially on a remote computer, or run entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any network form, such as a local area network (LAN) or a wide area network (WAN), or connected to an external computer (for example, via the Internet), or in a cloud computing environment, or used as a service (such as software as a service (SaaS)).

[0138] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this application, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this application. Although the above-mentioned application discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the embodiments of the application. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of the present application. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0139] Similarly, it should be noted that in order to simplify the description disclosed in this application and thus help understand one or more embodiments of the invention, in the above description of the embodiments of the application, multiple features are sometimes merged into one embodiment, figure or description thereof. However, this method of the application should not be interpreted as requiring more features than the features explicitly stated in each claim. On the contrary, the features of the subject matter claimed are less than all the features of the single embodiment disclosed above.

Claims

1. A distance measurement system, comprising: at least one storage device, the at least one storage device comprising a set of instructions; at least one processor in communication with the at least one storage device, wherein when executing the set of instructions, the at least one processor is configured to cause the system to: acquiring an image captured by a camera, the image including an object; For at least one of the at least two preset reference lines, Based on the map information, obtaining the height of the preset reference line; Select at least two reference points on the preset reference line; projecting the at least two reference points onto the image based on a horizontal reference distance from the preset reference line to the camera and the height of the preset reference line; Determining a prediction line in the image corresponding to the preset reference line based on at least two reference points of the projection; identifying at least one target line based on the object and at least one predicted line corresponding to at least one of the at least two preset reference lines; and Based on the at least one target line, a distance between the camera and the object is determined.

2. The system according to claim 1, characterized in that To identify at least one target line based on the object and at least one predicted line corresponding to at least one of the at least two preset reference lines, the at least one processor is configured to cause the system to: determining at least one measurement point associated with the object in the image; as well as The at least one target line in the image is identified based on the at least one measurement point and the at least one predicted line corresponding to at least one of the at least two preset reference lines.

3. The system according to claim 2, characterized in that To determine at least one measurement point associated with the object in the image, the at least one processor is configured to cause the system to: Determining a bounding box containing the object using an object detection algorithm; and Based on the bounding box, the at least one measurement point is determined.

4. The system according to claim 3, characterized in that The at least one processor is configured to cause the system to further: selecting a midpoint of a side of the bounding box as the at least one measurement point; Selecting at least one endpoint of the edge of the bounding box as the at least one measurement point; or One or more points are selected within the bounding box as the at least one measurement point.

5. The system according to any one of claims 2 to 4, characterized in that: The at least one measurement point is located on at least one of the at least one target lines in the image, or between the at least one target lines.

6. The system according to claim 1, characterized in that The at least two preset reference lines are perpendicular to the optical axis of the camera.

7. The system according to claim 6, characterized in that The at least two preset reference lines are equidistant or non-equidistant.

8. The system according to claim 1, characterized in that To identify at least one target line based on the object and at least one predicted line corresponding to at least one of the at least two preset reference lines, the at least one processor is configured to cause the system to: selecting at least one candidate line from the at least one predicted line based on the object; as well as The at least one target line is determined by performing an interpolation operation on the at least one candidate line.

9. The system according to claim 1, characterized in that To determine the distance between the camera and the object, the at least one processor is configured to cause the system to: For each of the at least one target line, Based on the map information, obtaining a height corresponding to the target line; Acquire a horizontal distance to the camera corresponding to the target line; as well as determining a target distance corresponding to the target line based on the height and the horizontal distance; as well as The distance between the camera and the object is determined based on at least one target distance corresponding to the at least one target line.

10. The system according to claim 9, characterized in that The at least one processor is configured to cause the system to further: The distance between the camera and the object is determined based on an average or a weighted average of the at least one target distance.

11. The system according to claim 9 or 10, characterized in that: The at least one processor is configured to cause the system to further: The distance between the camera and the object is determined based on the sum or weighted sum of the at least one target distance.

12. A distance measurement method implemented on a computing device, the computing device comprising at least one processor, at least one storage device, and a communication platform connected to a network, the method comprising: acquiring an image captured by a camera, the image including an object; For at least one of the at least two preset reference lines, Based on the map information, obtaining the height of the preset reference line; Select at least two reference points on the preset reference line; projecting the at least two reference points onto the image based on a horizontal reference distance from the preset reference line to the camera and the height of the preset reference line; Determining a prediction line in the image corresponding to the preset reference line based on at least two reference points of the projection; identifying at least one target line based on the object and at least one predicted line corresponding to at least one of the at least two preset reference lines; as well as Based on the at least one target line, a distance between the camera and the object is determined.

13. The method according to claim 12, characterized in that The identifying at least one target line based on the object and at least one predicted line corresponding to at least one of the at least two preset reference lines comprises: determining at least one measurement point associated with the object in the image; and The at least one target line in the image is identified based on the at least one measurement point and the at least one predicted line corresponding to at least one of the at least two preset reference lines.

14. The method according to claim 13, characterized in that The determining at least one measurement point associated with the object in the image comprises: Determining a bounding box containing the object using an object detection algorithm; and Based on the bounding box, the at least one measurement point is determined.

15. The method according to claim 14, further comprising: selecting a midpoint of a side of the bounding box as the at least one measurement point; selecting at least one endpoint of the edge of the bounding box as the at least one measurement point; or One or more points are selected within the bounding box as the at least one measurement point.

16. The method according to any one of claims 13 to 15, characterized in that The at least one measurement point is located on at least one of the at least one target lines in the image, or between the at least one target lines.

17. The method according to claim 12, characterized in that The at least two preset reference lines are perpendicular to the optical axis of the camera.

18. The method according to claim 17, characterized in that The at least two preset reference lines are equidistant or non-equidistant.

19. The method according to claim 12, characterized in that The identifying at least one target line based on the object and at least one predicted line corresponding to at least one of the at least two preset reference lines comprises: selecting at least one candidate line from the at least one predicted line based on the object; and The at least one target line is determined by performing an interpolation operation on the at least one candidate line.

20. The method according to claim 12, characterized in that The determining the distance between the camera and the object comprises: For each of the at least one target line, Based on the map information, obtaining a height corresponding to the target line; Acquire a horizontal distance to the camera corresponding to the target line; and determining a target distance corresponding to the target line based on the height and the horizontal distance; and The distance between the camera and the object is determined based on at least one target distance corresponding to the at least one target line.

21. The method according to claim 20, further comprising: The distance between the camera and the object is determined based on an average or a weighted average of the at least one target distance.

22. The method according to claim 20 or 21, further comprising: The distance between the camera and the object is determined based on the sum or weighted sum of the at least one target distance.

23. A distance measurement system, comprising: an acquisition module configured to acquire an image captured by a camera, wherein the image includes an object; The target line recognition module is configured as follows: For at least one of the at least two preset reference lines, Based on the map information, obtaining the height of the preset reference line; Select at least two reference points on the preset reference line; projecting the at least two reference points onto the image based on a horizontal reference distance from the preset reference line to the camera and the height of the preset reference line; Determining a prediction line in the image corresponding to the preset reference line based on at least two reference points of the projection; identifying at least one target line based on the object and at least one predicted line corresponding to at least one of the at least two preset reference lines; as well as The distance determination module is configured to determine the distance between the camera and the object based on the at least one target line.

24. The system according to claim 23, characterized in that The target line identification module is further configured to: determining at least one measurement point associated with the object in the image; and The at least one target line in the image is identified based on the at least one measurement point and the at least one predicted line corresponding to at least one of the at least two preset reference lines.

25. The system according to claim 24, characterized in that The target line identification module is further configured to: Determining a bounding box containing the object using an object detection algorithm; and Based on the bounding box, the at least one measurement point is determined.

26. The system according to claim 25, characterized in that The target line identification module is further configured to: selecting a midpoint of a side of the bounding box as the at least one measurement point; Selecting at least one endpoint of the edge of the bounding box as the at least one measurement point; or One or more points are selected within the bounding box as the at least one measurement point.

27. The system according to any one of claims 24 to 26, characterized in that The at least one measurement point is located on at least one of the at least one target lines in the image, or between the at least one target lines.

28. The system according to claim 23, characterized in that The at least two preset reference lines are perpendicular to the optical axis of the camera.

29. The system according to claim 28, characterized in that The at least two preset reference lines are equidistant or non-equidistant.

30. The system according to claim 23, wherein: The target line identification module is further configured to: selecting at least one candidate line from the at least one predicted line based on the object; and The at least one target line is determined by performing an interpolation operation on the at least one candidate line.

31. The system according to claim 23, characterized in that The distance determination module is further configured to: For each of the at least one target line, Based on the map information, obtaining a height corresponding to the target line; Acquire a horizontal distance to the camera corresponding to the target line; as well as determining a target distance corresponding to the target line based on the height and the horizontal distance; as well as The distance between the camera and the object is determined based on at least one target distance corresponding to the at least one target line.

32. The system according to claim 31, characterized in that The distance determination module is further configured to: The distance between the camera and the object is determined based on an average or a weighted average of the at least one target distance.

33. The system according to claim 31 or 32, characterized in that The distance determination module is further configured to: The distance between the camera and the object is determined based on the sum or weighted sum of the at least one target distance.

34. A non-transitory computer-readable storage medium comprising at least one set of instructions, wherein: When executed by at least one processor of a computing device, the at least one set of instructions instructs the at least one processor to perform the following actions: acquiring an image captured by a camera, the image including an object; For at least one of the at least two preset reference lines, Based on the map information, obtaining the height of the preset reference line; Select at least two reference points on the preset reference line; projecting the at least two reference points onto the image based on a horizontal reference distance from the preset reference line to the camera and the height of the preset reference line; Determining a prediction line in the image corresponding to the preset reference line based on at least two reference points of the projection; identifying at least one target line based on the object and at least one predicted line corresponding to at least one of the at least two preset reference lines; as well as Based on the at least one target line, a distance between the camera and the object is determined.

35. An autonomous driving vehicle comprising: A detection component, a planning component and a control component, wherein the planning component is configured to: acquiring an image captured by a camera, the image including an object; For at least one of the at least two preset reference lines, Based on the map information, obtaining the height of the preset reference line; Select at least two reference points on the preset reference line; projecting the at least two reference points onto the image based on a horizontal reference distance from the preset reference line to the camera and the height of the preset reference line; Determining a prediction line in the image corresponding to the preset reference line based on at least two reference points of the projection; identifying at least one target line based on the object and at least one predicted line corresponding to at least one of the at least two preset reference lines; as well as Based on the at least one target line, a distance between the camera and the object is determined.

Images