System for operating movable device

Abstract

The present disclosure provides a system for operating a movable device. A computer includes a processor and a memory including instructions executable by the processor to generate a current keyframe point cloud based on a stereoscopic image pair as a stereoscopic camera travels through a scene, and determining a similar view query matrix and a relative view query matrix based on the current key frame point cloud. A distance matrix and a relative view distance matrix may be generated by comparing the similar view query matrix and the relative view query matrix to a reference matrix. A relative pose between a stereo camera and a reference may be determined by performing sequence matching in the distance matrix and the relative perspective distance matrix to determine a minimum sequence to determine a position in the scene during travel of the stereo camera through the scene.

Description

Technical Field

[0001] The present disclosure relates to a system for operating a mobile device. Background Art

[0002] The computer may operate a system and / or device, including a vehicle, robot, drone, and / or object tracking system. Data, including images, may be acquired by sensors and processed using a computer to determine the position of the system relative to objects in an environment surrounding the system. The computer may use the position data to determine a trajectory for moving the system in the environment. The computer may then determine control data to be transmitted to system components to control the system components to move the system according to the determined trajectory. Summary of the invention

[0003] Systems (including vehicles, robots, drones, etc.) can be operated by acquiring sensor data about the environment surrounding the system and processing the sensor data to determine the position of the system in real-world coordinates. The determined real-world position data can be processed to determine a path on which to operate the system or part of the system. For example, a robot can determine the position of a gripper attached to a robot arm relative to a conveyor belt. The robot can use the determined position to determine a path to operate to move a part of the robot to grab a workpiece. The vehicle can determine the position of the vehicle relative to a road. The vehicle can use the determined position to operate the vehicle from a current position to a planned position while maintaining a predetermined distance from the edge of the road. Vehicle operation will be used as a non-limiting example of system position determination in this article.

[0004] Determining the position of a vehicle based on image data is referred to as visual place recognition (VPR) in the context of this document. VPR uses image data to determine whether the current location is included in a pre-existing set or database of reference locations. VPR may include at least two different types of reference databases. The first may be a multi-session reference database, where the reference database is pre-existing and can be geo-referenced. Geo-reference refers to an example in which global navigation satellite system (GNSS) data (optionally combined with inertial measurement unit (IMU) data) is used to generate a database of 3D positions including real-world coordinates. The second may be an in-session reference database, where a reference database of 3D positions is determined by the vehicle based on previous positions visited in a continuous driving session. The in-session reference database may or may not be geo-referenced. A non-geographically referenced database may provide a relative pose between a camera and a reference database of 3D positions. For example, the current camera position may be determined relative to the camera position when the camera is first turned on. A geo-referenced database provides absolute positioning, where the position may be determined relative to a global reference system. For example, the vehicle position may be determined based on latitude and longitude.

[0005] The example of VPR described herein includes determining the position of the vehicle when the vehicle returns along a route that has been determined based on data collected by a vehicle traveling in the opposite direction. This is referred to herein as VPR based on relative view. Existing techniques for determining VPR for relative view include forward and backward (360°) cameras or multiple cameras with wide field of view lenses, such as fisheye cameras or lidar sensors. Other VPR techniques rely on semantic segmentation of scenes using deep neural networks. Semantic segmentation refers to the technology of identifying and locating objects such as buildings and roads. Using deep neural networks to perform semantic segmentation may require determining extensive object label data for training deep neural networks. The technology described herein for VPR for relative view enhances VPR by determining the position of the vehicle based on a limited field of view (e.g., 180°) stereo camera. Using limited field of view stereo cameras reduces system complexity and the amount of computing resources, including the memory required for 360° cameras, fisheye cameras, or lidar sensors. The VPR technology for relative view described herein can eliminate the need for training data and reduce the computing resources required for deep neural network solutions.

[0006] VPR for relative perspective can also enhance the ability of the VPR to determine the position of the vehicle despite traveling along a route in a direction opposite to the direction in which the reference database was acquired. The techniques described herein for VPR can additionally operate with references collected at a similar perspective without requiring a priori knowledge of the reference perspective. An example of VPR for relative perspective enhancing vehicle operation can be an example in which GNSS data is temporarily unavailable due to interference from a tall building. VPR for relative perspective can locate the vehicle relative to a previously acquired geo-referenced reference database and allow the vehicle to be operated despite the lack of GNSS data.

[0007] A method is disclosed herein, the method comprising: generating a current keyframe point cloud based on a stereo image pair as a stereo camera travels through a scene; determining a similar view query matrix and a relative view query matrix based on the current keyframe point cloud; and generating a distance matrix and a relative view distance matrix by comparing the similar view query matrix and the relative view query matrix to a reference matrix. A relative pose between the stereo camera and a reference may be determined by performing sequence matching in the distance matrix and the relative view distance matrix to determine a minimum sequence, thereby determining a position in the scene during the travel of the stereo camera through the scene. A keyframe point cloud may be generated based on a stereo image pair by projecting pixel data from the stereo image pair into the keyframe point cloud based on a pose estimate of the stereo camera and camera intrinsic parameters from one or more of a left camera or a right camera of the stereo camera. The similar view query matrix and the relative view query matrix may be determined based on data points included in the current keyframe point cloud and distances above a ground plane of the data points.

[0008] The similarity view query matrix may be arranged in columns parallel to the direction of motion of the stereo camera and in rows perpendicular to the direction of motion of the stereo camera. The relative view query matrix may be determined by flipping the similarity view query matrix laterally and vertically by exchanging relative rows and columns of the similarity view query matrix. The similarity view query matrix and the relative view query matrix may be compared with a reference matrix to generate similarity view distance matrix columns and relative view distance matrix columns by shifting the similarity view query matrix and the relative view query matrix laterally and vertically relative to the reference matrix and comparing overlap interval values.

[0009] Sequence matching can be performed on similar view distance matrix columns and relative view distance matrix columns by searching for a diagonal line including a minimum value indicating a match. Reference keyframes can be generated by moving in a scene in a first direction while acquiring stereo image pairs and generating reference keyframes at equal displacements of the stereo camera. Real-world positions in a scene determined during the movement of a stereo camera through the scene can be generated based on a georeferenced reference database. The vehicle can be operated based on the position in the scene by determining a path polynomial. A pose estimate and a depth image can be determined based on the stereo images using stereo vision odometry. The elements of the distance matrix can be determined as the minimum cosine distance between the flattened overlapping blocks of the similar view query matrix and the relative view query matrix and the reference matrix. The cosine distance can be determined by the equation 1-(a·b) / (||a||||b||). The diagonal of the distance matrix including the minimum value indicating a match has a slope magnitude approximately equal to + / -1.

[0010] A computer-readable medium is also disclosed, the computer-readable medium storing program instructions for performing some or all of the above method steps. A computer programmed to perform some or all of the above method steps is also disclosed herein, the computer comprising a computer device, the computer device being programmed to: generate a current keyframe point cloud based on a stereo image pair as the stereo camera travels through a scene; determine a similar perspective query matrix and a relative perspective query matrix based on the current keyframe point cloud; and generate a distance matrix and a relative perspective distance matrix by comparing the similar perspective query matrix and the relative perspective query matrix with a reference matrix. A relative pose between the stereo camera and a reference may be determined by performing sequence matching in the distance matrix and the relative perspective distance matrix to determine a minimum sequence, thereby determining a position in the scene during the stereo camera's travel through the scene. A keyframe point cloud may be generated based on a stereo image pair by projecting pixel data from the stereo image pair into the keyframe point cloud based on a pose estimate of the stereo camera and camera intrinsic parameters from one or more of a left camera or a right camera of the stereo camera. The similar perspective query matrix and the relative perspective query matrix may be determined based on data points included in the current keyframe point cloud and the distance above the ground plane of the data points.

[0011] The instructions may include additional instructions, wherein the similar perspective query matrix may be arranged in columns parallel to the direction of motion of the stereo camera and rows perpendicular to the direction of motion of the stereo camera. The relative perspective query matrix may be determined by flipping the similar perspective query matrix laterally and longitudinally by exchanging the relative rows and columns of the similar perspective query matrix. The similar perspective query matrix and the relative perspective query matrix may be compared with a reference matrix to generate similar perspective distance matrix columns and relative perspective distance matrix columns by shifting the similar perspective query matrix and the relative perspective query matrix laterally and longitudinally relative to the reference matrix and comparing overlapping interval values. Sequence matching may be performed on the similar perspective distance matrix columns and the relative perspective distance matrix columns by searching for a diagonal line including a minimum value indicating a match. Reference key frames may be generated by acquiring stereo image pairs while traveling in a first direction in the scene and generating reference key frames at equal displacements of the stereo camera. A real-world position in the scene determined during the travel of the stereo camera through the scene may be generated based on a georeferenced reference database. A vehicle may be operated based on the real-world position in the scene by determining a path polynomial. A pose estimate and a depth image may be determined based on the stereo images using stereo visual odometry. The elements of the distance matrix can be determined as the minimum cosine distance between the flattened overlapping blocks of the similar view query matrix and the relative view query matrix and the reference matrix. The cosine distance can be determined by the equation 1-(a·b) / (||a||||b||). The diagonal of the distance matrix including the minimum value indicating a match has a slope magnitude approximately equal to + / -1. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a block diagram of an exemplary vehicle system.

[0013] Figure 2 is a diagram of an exemplary stereo camera.

[0014] Figure 3 is an illustration of an exemplary scene including a vehicle.

[0015] Figure 4 is an illustration of an exemplary scenario including data points.

[0016] Figure 5 is a diagram of exemplary keyframe generation.

[0017] Figure 6 is a diagram of an exemplary descriptor.

[0018] Figure 7 is an illustration of an exemplary visual place recognition (VPR) for a relative view system.

[0019] Figure 8 is a diagram of an exemplary location recognition system.

[0020] Fig. 9 is a graphical representation of an exemplary matrix overlay.

[0021] Fig.10 is a flow chart of an exemplary process for performing VPR for relative viewing angles.

[0022] Fig.11 is a flow chart of an exemplary process for operating a vehicle based on determining a vehicle pose based on a VPR for a relative viewing angle. DETAILED DESCRIPTION

[0023] Figure 1 1 is an illustration of a vehicle computing system 100. The vehicle computing system 100 includes a vehicle 110, a computing device 115 included in the vehicle 110, and a server computer 120 remote from the vehicle 110. One or more computing devices 115 of the vehicle 110 may receive data regarding the operation of the vehicle 110 from sensors 116. The computing devices 115 may operate the vehicle 110 based on the data received from the sensors 116 and / or data received from the remote server computer 120. The server computer 120 may communicate with the vehicle 110 via a network 130.

[0024] The computing device 115 includes a processor and memory such as are known. In addition, the memory includes one or more forms of computer-readable media and stores instructions that can be executed by the processor to perform various operations including as disclosed herein. For example, the computing device 115 may include programming to operate one or more of vehicle braking, propulsion (i.e., controlling the acceleration of the vehicle 110 by controlling one or more of an internal combustion engine, an electric motor, a hybrid engine, etc.), steering, climate control, interior lights and / or exterior lights, etc., and determine whether and when the computing device 115 (rather than a human operator) controls such operations.

[0025] The computing device 115 may include more than one computing device (i.e., controllers included in the vehicle 110 for monitoring and / or controlling various vehicle components, etc. (i.e., propulsion controller 112, brake controller 113, steering controller 114, etc.)), or be communicatively coupled to the more than one computing device via a vehicle communication bus as further described below. The computing device 115 is typically arranged for communication over a vehicle communication network (i.e., including a bus in the vehicle 110, such as a controller area network (CAN), etc.); the vehicle 110 network may additionally or alternatively include, for example, known wired or wireless communication mechanisms, such as Ethernet or other communication protocols.

[0026] The computing device 115 may transmit messages to and / or receive messages from various devices (i.e., controllers, actuators, sensors (including sensor 116), etc.) in the vehicle 110 via the vehicle network. Alternatively or additionally, where the computing device 115 actually includes multiple devices, the vehicle communication network may be used for communication between devices represented as computing device 115 in this disclosure. In addition, as mentioned below, various controllers or sensing elements (such as sensor 116) may provide data to the computing device 115 via the vehicle communication network.

[0027] Additionally, computing device 115 may be configured to communicate with remote server computer 120 (i.e., cloud server) via network 130 through vehicle-to-infrastructure (V2I) interface 111, as described below, which includes hardware, firmware, and software that permit computing device 115 to communicate with remote server computer 120 (i.e., cloud server) via a vehicle-to-infrastructure (V2I) interface 111, such as wireless Internet (WI- ) or cellular network to communicate with the remote server computer 120. The V2X interface 111 may therefore include a computer configured to utilize various wired and / or wireless networking technologies (i.e., cellular, The computing device 115 may be configured to communicate with other vehicles 110 through a V2X (vehicle to the outside world) interface 111 using a vehicle-to-vehicle (V2V) network formed between neighboring vehicles 110 on the basis of a mobile ad hoc network or through an infrastructure-based network (i.e., based on cellular communication (C-V2X) wireless communication cellular, dedicated short-range communication (DSRC) and / or similar communications). The computing device 115 also includes a non-volatile memory such as known. The computing device 115 can record data by storing the data in the non-volatile memory for later retrieval and transmission to the server computer 120 or the user mobile device 160 via the vehicle communication network and the vehicle to infrastructure (V2I) interface 111.

[0028] As already mentioned, programming for operating one or more vehicle 110 components (i.e., braking, steering, propulsion, etc.) without human operator intervention is typically included in instructions stored in memory and executable by a processor of the computing device 115. Using data received in the computing device 115 (i.e., sensor data from sensors 116, server computer 120, etc.), the computing device 115 can make various determinations and / or control various vehicle 110 components and / or operations. For example, the computing device 115 may include programming to adjust vehicle 110 operating behavior (i.e., the physical manifestation of vehicle 110 operation), such as speed, acceleration, deceleration, steering, etc., as well as strategic behavior (i.e., controlling operating behavior in a manner intended to achieve efficient traversal of a route), such as the distance between vehicles and / or the amount of time between vehicles, lane changes, minimum gaps between vehicles, left turn crossing path minimums, arrival times at specific locations, and the minimum time from arrival at an intersection to crossing an intersection (without a signal light).

[0029] The term controller as used herein includes a computing device that is typically programmed to monitor and / or control a particular vehicle subsystem. Examples include propulsion controller 112, brake controller 113, and steering controller 114. The controller may be an electronic control unit (ECU) such as is known, possibly including additional programming as described herein. The controller may be communicatively connected to computing device 115 and receive instructions from the computing device to actuate the subsystem in accordance with the instructions. For example, brake controller 113 may receive instructions from computing device 115 to operate the brakes of vehicle 110.

[0030] One or more controllers 112, 113, 114 for the vehicle 110 may include known electronic control units (ECUs) and the like, including, as non-limiting examples, one or more propulsion controllers 112, one or more brake controllers 113, and one or more steering controllers 114. Each of the controllers 112, 113, 114 may include a corresponding processor and memory and one or more actuators. The controllers 112, 113, 114 may be programmed and connected to a vehicle 110 communication bus, such as a controller area network (CAN) bus or a local interconnect network (LIN) bus, to receive instructions from a computing device 115 and control actuators based on the instructions.

[0031] Sensors 116 may include a variety of devices known to provide data via a vehicle communication bus. For example, a radar fixed to a front bumper (not shown) of vehicle 110 may provide a distance from vehicle 110 to the next vehicle in front of vehicle 110, or a global positioning system (GPS) sensor disposed in vehicle 110 may provide geographic coordinates of vehicle 110. For example, the distance provided by radar and / or other sensors 116 and / or the geographic coordinates provided by a GPS sensor may be used by computing device 115 to operate vehicle 110 autonomously or semi-autonomously.

[0032] The vehicle 110 is typically a ground-based vehicle 110 (i.e., a passenger car, a light truck, etc.) capable of autonomous and / or semi-autonomous operation and having three or more wheels. The vehicle 110 includes one or more sensors 116, a V2I interface 111, a computing device 115, and one or more controllers 112, 113, 114. The sensors 116 may collect data related to the vehicle 110 and the operating environment of the vehicle 110. By way of example and not limitation, the sensors 116 may include, for example, altimeters, cameras, LIDAR, radars, ultrasonic sensors, infrared sensors, pressure sensors, accelerometers, gyroscopes, temperature sensors, Hall sensors, optical sensors, voltage sensors, current sensors, mechanical sensors (such as switches), etc. The sensors 116 may be used to sense the operating environment of the vehicle 110, i.e., the sensors 116 may detect phenomena such as weather conditions (rainfall, outside temperature, etc.), road slope, road position (i.e., using road edges, lane markings, etc.), or the position of target objects (such as neighboring vehicles 110). The sensors 116 may also be used to collect data, including dynamic vehicle 110 data related to the operation of the vehicle 110 , such as speed, yaw rate, steering angle, engine speed, brake pressure, oil pressure, power levels applied to controllers 112 , 113 , 114 in the vehicle 110 , connectivity between components, and accurate and timely performance of components of the vehicle 110 .

[0033] The server computer 120 generally has features in common with the V2I interface 111 and computing device 115 of the vehicle 110, such as a computer processor and memory and configuration for communicating via the network 130, and therefore these features will not be further described to reduce redundancy. The server computer 120 can be used to develop and train software that can be transferred to the computing device 115 in the vehicle 110.

[0034] Figure 2 1 is a diagram of a stereo camera 200. The stereo camera 200 can acquire data that can be processed by the computing device 115 to determine the distance to the position in the field of view of the stereo camera 200 for input to the VPR system. Determining the vehicle position based on the position in the field of view of the camera is called visual odometry. Visual odometry uses image data to estimate the environment around the camera and determine the trajectory of the camera within the structure / environment. The stereo camera 200 generates a point cloud of three-dimensional (3D) positions, which can be used to determine the position of objects in the environment around the vehicle 110 and the position of the vehicle 110 relative to those objects. The VPR system can use the depth data generated by the visual odometry to determine the best match between the acquired point cloud data points and the data points in the previously acquired 3D map of the environment in which the vehicle 110 is traveling. The 3D map data points can be determined by traveling along the route and acquiring the stereo camera 200 point cloud data points and assembling them into a 3D map. The VPR system can determine the position of the stereo camera 200 relative to the 3D map. In examples where the locations included in the 3D map are geo-referenced and known in real-world coordinates, the VPR system can determine the location of the stereo camera in real-world coordinates.

[0035] The stereo camera 200 includes a left camera 202 and a right camera 204, both of which are configured to view a scene 206. The left camera 202 and the right camera 204 each include an optical center 208, 210, respectively. The left camera 202 and the right camera 204 can be configured to view the scene 206 along parallel optical axes 224, 226, respectively. The optical axes 224, 226 and the sensor plane 218 can be configured to be perpendicular, thereby forming a z-axis and an x-axis. The x-axis can be defined as passing through the optical center 208 of one of the stereo cameras 202, 204. The y-axis is in a direction perpendicular to the page. The sensor plane 218 includes a left image sensor 214 and a right image sensor 216, respectively included in the left camera 202 and the right camera 204. The optical centers 208, 210 are separated by a baseline distance b. The left image sensor 214 and the right image sensor 216 are located at focal lengths 220, 222 of the optical centers 208, 210, respectively, in the sensor plane 218. For ease of illustration, the left image sensor 214 and the right image sensor 216 are shown in a virtual image plane, for example, in front of the optical centers 208, 210. In reality, the left image sensor 214 and the right image sensor 216 would be behind the optical centers 208, 210, for example, at Figure 2 To the left of the optical centers 208, 210.

[0036] The stereo camera 200 can be used to determine the depth z of a point P in the scene 206 from a plane defined relative to the optical centers 208, 210 of the left camera 202 and the right camera 204, respectively. The distance x from the right optical axis 226 to the point P is p 228. Assuming the pinhole optical model of the left camera 202 and the right camera 204, the image of point P is at point x l and x r The value x is projected onto the left image sensor 214 and the right image sensor 216. l and x r Indicates the distance from the left optical axis 224 and the right optical axis 226. The value (x l -x r )=d is called stereo disparity d. The stereo disparity d can be converted from pixel data to real-world distance by multiplying by a scale factor s. The depth z can be determined from the stereo disparity d, focal length f, pixel scale s, and baseline b by the following equation:

[0037]

[0038] In this example, the stereo image pair is displaced from each other only in the x-direction along the X-axis, which means that the stereo disparity d is determined only along the X-axis.

[0039] The depth z measurements from the stereo image pair can be used to determine the pose of the stereo camera 200 using visual odometry. Visual odometry uses a collection of depth z measurements from the stereo image pair and camera intrinsic parameters to determine the pose of the stereo camera 200 relative to the scene 206. The camera intrinsic parameters include the camera focal length in the x and y directions, the camera sensor scale in the x and y directions, and the optical center of the camera optics relative to the sensor. The camera intrinsic parameters can be indicated in pixels or in real-world units. After the depth estimate has been made and associated with the pixels in the left camera 202 to form a depth image, the 3D data points can be projected into the 3D space with the left camera intrinsic parameters to form a depth image.

[0040] Figure 3 302 is an illustration of a scene 302. The scene 302 includes a vehicle 110 traveling through the scene 302 on a route 304 indicated by an arrow. The vehicle 110 includes two cameras forming a single stereo camera 306 that acquires overlapping stereo image pairs 310, 312, 314, 316, 318, 320 of the scene 302 as the vehicle travels along the route 304. Figure 2 Overlapping stereo image pairs 310 , 312 , 314 , 316 , 318 , 320 are processed as described to determine a depth image based on the locations of data points in scene 302 .

[0041] The VPR technique for relative perspective as described herein includes first mapping a route in a vehicle 110 equipped with one or more stereo cameras 200 by traveling along the route in a first direction. The stereo cameras 200 acquire stereo image pairs 310, 312, 314, 316, 318, 320 of a scene 302 along the route 304. As described below with respect to Figures 4 to 6 The stereo image pair is processed as described to determine a reference descriptor database. The reference descriptor database is a database that includes descriptors acquired while the vehicle 110 is traveling in a first direction along the route 304, the descriptors being matrices describing the scene 302 along the route 304. The VPR techniques for relative perspective as described herein allow the vehicle 110 to travel along the route 304 in a direction opposite to the direction in which the reference descriptor database was acquired to determine the real-world position of the vehicle 110 using the reference descriptor database.

[0042] Figure 4 is an illustration of scenario 402 showing that by utilizing Figure 2The described stereo visual odometry software program processes data points 404, 406 determined from a stereo image pair acquired by a vehicle 110 traveling in a scene 402. The stereo visual odometry software executes on a computing device 115 and outputs vehicle 110 pose and 3D position data for the data points 404, 406. The data points 404, 406 may be acquired to describe features such as buildings, foliage, or structures (such as bridges) adjacent to a road, and may be used by the VPR techniques for relative view described herein to determine the position of the vehicle 110 as it travels through the scene 402 in a direction opposite to the direction in which the data points 404, 406 were acquired.

[0043] Figure 5 is a diagram showing keyframe generation based on the depth image output from the stereo visual odometry software to accumulate data points 404, 406. The keyframe generation rejects noisy data points 404, 406 and determines when to create new overlapping keyframes 502, 504, 506, 508, 510 (individually, and collectively, keyframe 516). The coordinate system of the keyframe 516 is centered on the vehicle 110, and the keyframes 516 can overlap. Stereo triangulation error increases approximately quadratically with depth, so as an initial step to reject noisy data points 404, 406, pixels with a depth exceeding a user-selected threshold rd are discarded. Each depth image from the stereo visual odometry software has a pose estimate for the stereo camera. The pose estimate of the stereo camera pair and the camera intrinsic and extrinsic parameters are used to project each valid data point 404, 406 of the depth image into a common world coordinate system. The common world coordinate system refers to the reference system in which the visual odometry pose estimate is represented. A common world coordinate system is defined as an initial camera pose, comprising the position and orientation (pose) indicated by the first image processed by the stereo odometry system.

[0044] Keyframes 516 indicating equal displacements of the stereo cameras are generated at constant distances apart to aid in subsequent matching with the acquired data points 404, 406. The keyframes 516 may overlap while remaining spaced a constant distance apart. A path distance traveled by the vehicle 110 along the route 304 as the vehicle 110 acquired the data points 404, 406 is determined. The path distance is the sum of the Euclidean distances between the stereo vision odometry software position estimates since the last keyframe 516 was generated. When the path distance exceeds the desired descriptor spacing s, a new keyframe 516 is generated to include a point cloud based on the data points 404, 406 centered on the current vehicle pose.

[0045] To create a new keyframe point cloud, programming may be performed to transform the accumulated point cloud to the current camera frame and select all data points 404, 406 within a horizontal radius rk around the camera position. Each time a new keyframe 516 is generated, the second largest horizontal radius ra is used to eliminate distant data points 404, 406 in the accumulated point cloud data points 512. A new keyframe 516 including point cloud data points 512 is not created until the path distance exceeds a threshold, such as 1.5rk, with the goal that the keyframe point cloud 512 will be well populated with data points 404, 406. Coordinate axes 514 indicate x, y, and z directions, where z is the direction of forward longitudinal motion of the vehicle 110 along the route 304, x is a direction transverse to the direction of motion of the vehicle 110, and y is the height of the point cloud data point 512 measured relative to the optical center of the camera 202, assuming the height is perpendicular to the x, z ground plane. The point cloud data point indicates a 3D position in the current camera reference system.

[0046] During descriptor formation, the height of the camera above the ground is added to the y coordinate of the point to calculate the height of the point above the x,z ground plane. The height of the camera 200 above the x,z ground plane can be determined when the camera 200 is installed in the vehicle 110. The keyframes acquired as the stereo camera travels through the scene for determining the reference database are referred to herein as reference keyframes. The keyframes acquired by the stereo camera to position the stereo camera relative to the previously acquired reference database are referred to herein as current keyframes.

[0047] Figure 6 is an illustration of a descriptor 600. Descriptor 600 is an N×M matrix describing a rectangular region along route 304, wherein the columns of the N×M matrix are parallel to route 304 and the rows of the N×M matrix are perpendicular to route 304. Point cloud data points 512 included in the rectangular region indicated by the entry in the N×M matrix are represented by the maximum value (highest) point cloud data point 512. For each key frame 516 portion of point cloud data points 512, descriptor 600 is formed. Since the descriptor 600 is formed by the above-mentioned Figure 2 and Figure 3 The data points 404, 406 of the point cloud data points 512 obtained by the discussed visual odometry method have an absolute scale in real-world coordinates, so the descriptor 600 can be generated based on the point cloud data points 512 of the key frame 516. The descriptor 600 is created from the key frame 516 portion of the point cloud data points 512, which appear in a 2rlo×2rla meter horizontal rectangle centered at the origin 604 of the camera frame, where the 2rlo meter side is aligned with the longitudinal direction (or forward direction z) and the 2rla meter side is aligned with the lateral direction (x).

[0048] A rectangular domain centered at the origin 604 of the camera frame is divided into m equal-sized rows along the longitudinal axis and into n equal-sized columns along the transverse axis to create intervals 602. The maximum distance above the ground plane of the x, y, z data point 512 captured within each interval 602 determines the interval value for each of the mxn intervals included in the mxn descriptor 600 matrix. If there is no data point 512 within the interval, the corresponding value in the descriptor 600 is set to zero. The height above the ground of the data point 512 = [x, y, z] is calculated as hp = hc-y, where hc is the known height of the left camera above the ground. To ensure that the rectangular domain of the descriptor 600 is completely filled with point 512, the following relationship should be satisfied:

[0049] rd>=rlo and rk>=r2lo+r2la (2)

[0050] Where rd is the threshold for rejecting noisy data points 512, and rk is the horizontal radius within which the data points 512 are taken.

[0051] The vehicle 110 including one or more stereo cameras 306 may travel along the route 304, wherein the one or more stereo cameras acquire data points 404, 406, which may be processed using visual odometry to determine depth data and camera pose data, which may be described in detail with respect to FIG. Figures 4 to 6 Processing is performed as described to determine a plurality of descriptors 600 that form a reference descriptor database describing a scene 302 of a vehicle 110 traveling in a first direction along a route 304 .

[0052] Figure 7 is an illustration of a VPR system 700 for relative view for obtaining a relative pose between a camera and a reference database. The VPR system for relative view is generally a software program that can be executed on a computing device 115 included in a vehicle 110. The VPR system for relative view determines the position of the vehicle as the vehicle 110 travels on a route 304 that has been previously mapped by a vehicle 110 traveling in an opposite direction on the same route 304. Mapping the route 304 includes determining a reference descriptor database by determining a plurality of descriptors 600 from stereo visual odometry data, as described above with respect to FIG. Figures 2 to 6 described.

[0053] The VPR system 700 for relative view begins by acquiring one or more stereo image pairs using a stereo camera 702, the stereo camera including a left camera 704 and a right camera 706. The stereo camera image pairs are input to a stereo visual odometry method 708 to determine a depth image 712 and a pose 710 of the stereo camera 702. The depth image 712 and the pose 710 are input to a place recognition system 716 along with a reference descriptor database. The place recognition system 716 generates descriptors 600 based on the depth image 712 and the pose 710, and uses the descriptors 600 to perform variable offset dual descriptor 600 distance calculations and dual distance matrix sequence matching according to the reference descriptor database, as described below with respect to Figure 8 The place recognition system 716 determines the descriptor 600 with the lowest matching score from the dual distance matrix sequence matching from the reference descriptor database, selects it as the best matching location, and outputs the best matching descriptor 600 as the final match 718 .

[0054] Figure 8 1 is an illustration of a place recognition system 716, which may include a software program that may be executed on a computing device 115 included in a vehicle 110. The place recognition system 716 may receive as input a current keyframe point cloud 802 generated by stereo visual odometry based on a depth image in a common world coordinate system received by a descriptor generator 806. The descriptor generator 806 generates the descriptors based on the current keyframe point cloud 802 as described above with respect to FIG. Figure 6 The descriptor 600 is then arranged in columns parallel to the direction of motion of the stereo camera and rows perpendicular to the direction of motion of the stereo camera, and a similarity perspective query matrix 808 is then determined. The similarity perspective query matrix 808 is passed to a double flip processor 836 which performs a double flip on the rows and columns of the similarity perspective query matrix 808 by flipping the similarity perspective query matrix 808 laterally and vertically by swapping the relative rows and columns of the similarity perspective query matrix 808, thereby forming a relative perspective query matrix 824. This has the effect of generating a relative perspective query matrix 824 that appears as if it was generated by a vehicle 110 traveling along the route 304 in the same direction as the vehicle 110 that generated the reference descriptor database.

[0055] The similar view query matrix 808 and the relative view query matrix 824 include intervals (i.e., metadata) at each row position and column position, the intervals including the value of the maximum height data point in the current key frame point cloud data point 512 included in the x, z address indicated by the intervals. At the similar descriptor distance calculation 810 and the relative descriptor distance calculation 826, the descriptors 600 included in the similar view query matrix 808 and the relative view query matrix 824 are compared with the descriptors 600 from the reference descriptor database 838 to form similar view distance matrix columns 812 and relative view distance matrix columns 828. The values ​​of the intervals included in the similar view query matrix 808 and the relative view query matrix 824 are compared with the intervals in the descriptors 600 included in the reference descriptor database 838.

[0056] The similar view distance matrix 814 and the relative view distance matrix 830 may be determined by calculating separate distances. Let Q∈R,m×n be the query descriptor 600 matrix, and R∈R,m×n be the descriptor 600 matrix from the reference descriptor database 838. Additionally, let Q[i,j,h,w] denote the submatrix obtained by selecting rows {i,...,i+h-1} and columns {j,...,j+w-1} from Q. The descriptor 600 distance between Q and R is then calculated as:

[0057] d(Q, R)=min{k∈slo, l∈sla}cd(Q[iQ, jQ, h, w], R[iR, jR, h, w]) (3)

[0058] in

[0059] iQ=max(1,-k+1), jQ=max(1,-l+1) (4)

[0060] iR=max(1,k+1), jR=max(1,l+1) (5)

[0061] and

[0062] h=m-|k|,w=n-|l| (6)

[0063] where slo and sla are the sets of longitudinal and lateral shifts, respectively, and cd(A, B) is the cosine distance between matrices A and B, defined as:

[0064] cd(A,B)=1-(a·b) / (||a||||b||)(7)

[0065] Where a and b are vectors obtained by flattening matrices A and B, respectively. Similar view distance matrix column 812 and relative view distance matrix column 828 are output and collected into similar view distance matrix 814 and relative view distance matrix 830, respectively.

[0066] Fig. 9 is a diagram showing the overlap region between matrices Q and R, where cosine distances are applied to a single longitudinal and lateral shift to produce a similar view distance matrix 814 and a relative view distance matrix 830. The descriptor distances determined as described above are invariant to horizontal shifts, but are not invariant to rotations. The relative view case is taken into account by doubly flipping the reference descriptors 600 (i.e., once about each axis). Doubly flipping the descriptors 600 produces a descriptor 600 that is similar to a descriptor that had been generated from the relative view. The double flipping is performed on the query descriptors 600 rather than the reference descriptors 600 to prevent doubling the reference database size or requiring a double flip of the reference for each new query. For increased efficiency, calculations across separate reference descriptors 600 may be performed in parallel.

[0067] Back to Figure 8 , receiving the distances calculated for the similar view distance matrix (Dsim) 814 and the relative view distance matrix (Dopp) 830 through sequence matching 816, 832, respectively. The descriptor distances calculated with the original query help capture the distance matrix of the similar view Dsim 814, while the descriptor distances calculated with the double flipped query help capture the distance matrix of the relative view Dopp 830. It can be expected that the similar view sequence matching appears as a line with a positive slope in the similar view Dsim 814 and does not produce a pattern in the relative view Dopp 830. Conversely, it can be expected that the relative view sequence matching appears as a line with a negative slope in the relative view Dopp 830 and does not produce a pattern in the similar view Dsim 814. In addition, the formation of the descriptor 600 described above will ensure that the slope magnitude is approximately equal to 1 in either case.

[0068] In order to predict the correct match without any prior knowledge about the view angle, sequence matching 816, 832 is performed separately within each distance matrix Dsim 814, Dopp 830. For example, in the past w queries, sequence matching 816 can be performed for similar view angles Dsim 814 with a positive slope, and sequence matching 832 can be performed for relative view angles Dopp 830 with a negative slope. Sequence matching 816, 832 process output sequence matching 818, 834, respectively.

[0069] In each example, we evaluate the slope in the distance matrix 814, 830 ranging from vmin to vmax, where vmin is slightly less than 1 and vmax is slightly greater than 1, for example vmin = 0.9 and vmax = 1.1. For efficiency, the sum of multiple candidates can be calculated in parallel. Each of the two searches returns a predicted match for the query at the center of the search window and a score. The matches 818, 834 are compared at the match comparison 820, and the match with the lowest score is output at the match output 822. The best match output at the match output 822 indicates the best descriptor 600 from the reference descriptor database 838. The lateral shift and longitudinal shift used to form the best match are combined with the real-world position of the best matching descriptor 600 included in the reference descriptor database 838 and the pose of the vehicle 110 from the descriptor 600 used to form the similar perspective query matrix 808 and the relative perspective query matrix 824 to determine the best estimate of the vehicle pose.

[0070] Fig.10 1 is a flow chart of a process 1000 for performing VPR for relative perspective. For example, the process 1000 may be implemented in a computing device 115 included in a vehicle 110. The process 1000 includes multiple blocks that may be performed in the order shown. Alternatively or additionally, the process 1000 may include fewer blocks, or may include blocks that are performed in a different order.

[0071] Process 1000 begins at box 1002, where one or more stereo cameras 702 included in the vehicle 110 acquire one or more stereo image pairs from the scene 302, 402 while the vehicle 110 is traveling on the route 304 in a direction opposite to the direction in which the vehicle 110 previously traveled along the route 304 to acquire a reference descriptor database 714 based on reference key frames describing locations in the scene.

[0072] At block 1004, one or more stereo image pairs are received by stereo visual odometry 708 to determine a depth image 712 in the scene 302, 402. The depth image 712 and a stereo camera pose 710 including intrinsic and extrinsic camera parameters are combined to form a point cloud in a common world coordinate system.

[0073] At block 1006 , the place recognition system 716 generates a descriptor 600 based on the current keyframe point cloud 802 and determines the descriptor 600 to include a similar view query matrix 808 and a relative view query matrix 824 .

[0074] At box 1008, the similar view query matrix 808 and the relative view query matrix 824 are compared to the descriptors 600 from the reference descriptor database 838 to form similar view distance matrix columns 812 and relative view distance matrix columns 828 that contribute to the similar view distance matrix 814 and the relative view distance matrix 830, respectively.

[0075] At block 1010, the place recognition system 716 performs the same analysis as described above with respect to the similarity view distance matrix 814 and the relative view distance matrix 830. Figure 7 The described sequence matching is performed to determine how well the descriptors 600 from the reference database 838 match the similar view query matrix 808 and the relative view query matrix 824. The sequences of values ​​in the similar view distance matrix 814 and the relative view distance matrix 830 are processed by sequence matching to determine the sequence of values ​​that produces the best match to a line with a slope of approximately + / - 1.

[0076] At block 1012 , the matches 818 , 834 output by the sequence matches 816 , 832 are compared at the match comparison 820 to determine which match has the lowest value to be output at the match output 822 .

[0077] At block 1014, a best estimated vehicle pose is determined based on the best matching position in the reference descriptor database 838, the lateral offset and longitudinal offset of the best matching descriptor 600, and the output. After block 1014, process 1000 ends.

[0078] Fig.11 1 is a flow chart of a process 1100 for operating a vehicle 110 based on a best estimate of the vehicle pose according to a VPR for a relative view. As a non-limiting example, the process 1100 is described in terms of operating a vehicle 110. The example describes a VPR for a relative view applied to a multi-session reference database 838, which includes a geo-reference as described above. The VPR for a relative view may also be applied to a session reference database 838 with or without a geo-reference. In examples where a geo-reference is not included in the reference database 838, a separate process may be used to obtain real-world coordinates. The process 1100 may be more generally applied to mobile systems. For example, the process 1100 may provide high-resolution pose data to mobile systems such as mobile robots and drones. The process 1100 may also be applied to systems including mobile components, such as fixed robots, package sorting systems, and security systems. The process 1100 may be implemented by a computing device 115 included in the vehicle 110. The process 1100 includes a plurality of blocks that may be executed in the order shown. Alternatively or additionally, process 1100 may include fewer blocks, or may include blocks performed in a different order.

[0079] Process 1100 begins at block 1102, where a computing device 115 in a vehicle 110 determines a vehicle pose based on a stereo image pair and a reference descriptor database 838. The reference descriptor database 838 may be generated by a vehicle 110 traveling in a first direction on route 304, and the stereo image pair may be generated by a vehicle 110 traveling in a direction opposite to the direction in which the vehicle 110 generating the reference descriptor database 838 was traveling, as described above with respect to Figures 2 to 9 as described.

[0080] At box 1104, the computing device 115 determines a path polynomial that directs vehicle motion from the current position based on the determined vehicle pose. As will be appreciated, the vehicle 110 may be operated by determining a path polynomial function that maintains, for example, minimum and maximum limits on lateral and longitudinal accelerations.

[0081] At block 1106, the computing device 115 operates the vehicle 110 based on the determined path polynomial by transmitting commands to the controllers 112, 113, 114 to control one or more of vehicle propulsion, steering, and braking according to an appropriate algorithm. In one or more examples, data based on the path polynomial may alternatively or additionally be transmitted to a human-machine interface, such as a display, in the vehicle 110, for example. After block 1106, the process 1100 ends.

[0082] Computing devices such as those described herein typically each include commands that can be executed by one or more computing devices such as those identified above and used to implement the blocks or steps of the processes described above. For example, the process blocks described above may be embodied as computer executable commands.

[0083] The computer executable instructions may be compiled or interpreted by a computer program created using a variety of programming languages ​​and / or techniques, including but not limited to the following in single or combined form: Java TM , C, C++, Python, Julia, SCALA, Visual Basic, Java Script, Perl, HTML, etc. Typically, a processor (i.e., a microprocessor) receives commands from a memory, a computer-readable medium, etc., and executes these commands to perform one or more processes including one or more of the processes described herein. Such commands and other data may be stored in files and transmitted using a variety of computer-readable media. Files in a computing device are typically collections of data stored on a computer-readable medium such as a storage medium, a random access memory, etc.

[0084] Computer-readable media (also referred to as processor-readable media) include any non-transitory (i.e., tangible) media that participate in providing data (i.e., instructions) that can be read by a computer (i.e., by a processor of a computer). Such media can take many forms, including, but not limited to, non-volatile media and volatile media. Instructions can be transmitted via one or more transmission media, including optical fibers, wires, wireless communications, including internals that make up a system bus coupled to a processor of a computer. Common forms of computer-readable media include, for example, RAM, PROM, EPROM, FLASH-EEPROM, any other memory chip or cassette, or any other medium from which a computer can read.

[0085] Unless otherwise expressly indicated herein, all terms used in the claims are intended to be given their ordinary and customary meanings as understood by those skilled in the art. Specifically, unless a claim recites an express limitation to the contrary, the use of singular articles such as "a," "an," "the," and "said" should be interpreted as reciting one or more of the indicated elements.

[0086] The term "exemplary" is used herein in the sense of referring to an example, ie, reference to "exemplary widget" should be interpreted as referring merely to an example of a widget.

[0087] The adverb “approximately” modifying a value or result means that the shape, structure, measurement, value, determination, calculation, etc. may vary from the exactly described geometry, distance, measurement, value, determination, calculation, etc. due to imperfections in materials, machining, manufacturing, sensor measurement, calculation, processing time, communication time, etc.

[0088] In the accompanying drawings, like reference numerals indicate like elements. With respect to the media, processes, systems, methods, etc. described herein, it should be understood that although the steps or blocks of such processes, etc. have been described as occurring in a sequence in a particular order, such processes may be practiced by the described steps being performed in an order other than that described herein. It should also be understood that certain steps may be performed simultaneously, other steps may be added, or certain steps described herein may be omitted. In other words, the description of the processes herein is provided for the purpose of illustrating certain embodiments and should in no way be construed as limiting the claimed invention.

[0089] According to the present invention, a system is provided, the system having: a computer, the computer including a processor and a memory, the memory including instructions, the instructions executable by the processor to: generate a current key frame point cloud based on a stereo image pair when a stereo camera moves through a scene; determine a similar perspective query matrix and a relative perspective query matrix based on the current key frame point cloud; generate a distance matrix and a relative perspective distance matrix by comparing the similar perspective query matrix and the relative perspective query matrix with a reference matrix; and determine a relative pose between the stereo camera and a reference by performing sequence matching in the distance matrix and the relative perspective distance matrix to determine a minimum sequence to determine a position in a scene during the stereo camera moving through the scene.

[0090] According to one embodiment, the instructions include further instructions for generating a keyframe point cloud based on a stereo image pair by projecting pixel data from the stereo image pair into a keyframe point cloud based on a pose estimate of the stereo camera and camera intrinsic parameters from one or more of a left camera or a right camera of the stereo camera.

[0091] According to one embodiment, the instructions include further instructions for determining a similarity view query matrix and a relative view query matrix based on data points included in the current keyframe point cloud and distances above a ground plane of the data points.

[0092] According to one embodiment, the similarity view query matrix is ​​arranged with columns parallel to the moving direction of the stereo camera and rows perpendicular to the moving direction of the stereo camera.

[0093] According to one embodiment, the relative view query matrix is ​​determined by flipping the similarity view query matrix horizontally and vertically by swapping relative rows and columns of the similarity view query matrix.

[0094] According to one embodiment, the instructions include further instructions for comparing the similar perspective query matrix and the relative perspective query matrix with a reference matrix to generate similar perspective distance matrix columns and relative perspective distance matrix columns by shifting the similar perspective query matrix and the relative perspective query matrix laterally and vertically relative to the reference matrix and comparing overlapping interval values.

[0095] According to one embodiment, the instructions comprise further instructions for performing sequence matching on the similar view distance matrix columns and the relative view distance matrix columns by searching for a diagonal line comprising a minimum value indicating a match.

[0096] According to one embodiment, the instructions include further instructions for generating reference keyframes by travelling in a first direction in the scene while acquiring stereoscopic image pairs and generating reference keyframes at equal displacements of the stereo cameras.

[0097] According to one embodiment, the instructions comprise further instructions for determining, based on a geo-referenced reference database, real-world locations in the scene determined during travel of the stereo camera through the scene.

[0098] According to one embodiment, the instructions include further instructions for operating the vehicle based on the position in the scene by determining a path polynomial.

[0099] According to the present invention, a method is provided, the method comprising: generating a current key frame point cloud based on a stereo image pair when a stereo camera moves through a scene; determining a similar view query matrix and a relative view query matrix based on the current key frame point cloud; generating a distance matrix and a relative view distance matrix by comparing the similar view query matrix and the relative view query matrix with a reference matrix; and determining a relative pose between the stereo camera and a reference by performing sequence matching in the distance matrix and the relative view distance matrix to determine a minimum sequence to determine a position in the scene during the stereo camera moving through the scene.

[0100] In one aspect of the present invention, the method includes: generating a keyframe point cloud based on a stereo image pair by projecting pixel data from the stereo image pair into a keyframe point cloud based on a pose estimate of the stereo camera and camera intrinsic parameters from one or more of a left camera or a right camera of the stereo camera.

[0101] In one aspect of the invention, the method includes determining a similarity view query matrix and a relative view query matrix based on data points included in a current keyframe point cloud and distances above a ground plane of the data points.

[0102] In one aspect of the present invention, the similarity view query matrix is ​​arranged with columns parallel to the motion direction of the stereo camera and rows perpendicular to the motion direction of the stereo camera.

[0103] In one aspect of the invention, the relative view query matrix is ​​determined by flipping the similarity view query matrix horizontally and vertically by swapping relative rows and columns of the similarity view query matrix.

[0104] In one aspect of the present invention, the method includes: comparing a similar perspective query matrix and a relative perspective query matrix with a reference matrix to generate similar perspective distance matrix columns and relative perspective distance matrix columns by shifting the similar perspective query matrix and the relative perspective query matrix horizontally and vertically relative to the reference matrix and comparing overlapping interval values.

[0105] In one aspect of the invention, the method comprises performing sequence matching on similar view distance matrix columns and relative view distance matrix columns by searching for a diagonal line comprising a minimum value indicating a match.

[0106] In one aspect of the invention, the method comprises generating reference keyframes by travelling in a first direction in a scene while acquiring stereoscopic image pairs and generating reference keyframes at equal displacements of the stereo cameras.

[0107] In one aspect of the invention, the method comprises determining, based on a geo-referenced reference database, real-world locations in a scene determined during travel of a stereo camera through the scene.

[0108] In one aspect of the invention, the method includes operating a vehicle based on a position in a scene by determining a path polynomial.

Claims

1. A method, comprising: generating a current keyframe point cloud based on the stereo image pair as the stereo camera travels through the scene; Determine a similar perspective query matrix and a relative perspective query matrix based on the current key frame point cloud; Generate a distance matrix and a relative perspective distance matrix by comparing the similar perspective query matrix and the relative perspective query matrix with a reference matrix; as well as A relative pose between a stereo camera and a reference is determined by performing sequence matching in the distance matrix and the relative view distance matrix to determine a minimum sequence to determine a position in the scene during a movement of the stereo camera through the scene.

2. The method of claim 1 further comprises generating the key frame point cloud based on the stereo image pair by projecting pixel data from the stereo image pair into the key frame point cloud based on a pose estimate of the stereo camera and camera intrinsic parameters from one or more of a left camera or a right camera of the stereo camera.

3. The method of claim 1, further comprising determining the similarity view query matrix and the relative view query matrix based on data points included in the current key frame point cloud and distances above a ground plane of the data points. 4 . The method of claim 1 , wherein the similarity view query matrix is ​​arranged with columns parallel to a movement direction of the stereo camera and rows perpendicular to the movement direction of the stereo camera. 5 . The method of claim 1 , wherein the relative view query matrix is ​​determined by flipping the similarity view query matrix horizontally and vertically by swapping relative rows and columns of the similarity view query matrix.

6. The method as claimed in claim 1 also includes comparing the similar perspective query matrix and the relative perspective query matrix with the reference matrix to generate similar perspective distance matrix columns and relative perspective distance matrix columns by shifting the similar perspective query matrix and the relative perspective query matrix horizontally and vertically relative to the reference matrix and comparing overlapping interval values.

7. The method of claim 1, further comprising performing sequence matching on the similar view distance matrix columns and the relative view distance matrix columns by searching for a diagonal line including a minimum value indicating a match.

8. The method of claim 1, further comprising generating the reference keyframe by traveling in the scene in a first direction while acquiring stereoscopic image pairs and generating reference keyframes at equal displacements of the stereo cameras.

9. The method of claim 1, further comprising determining real-world locations in the scene determined during travel of the stereo camera through the scene based on a geo-referenced reference database.

10. The method of claim 1 further comprising operating a vehicle based on the position in the scene by determining a path polynomial.

11. The method of claim 1 , wherein a pose estimate and a depth image are determined based on the stereo images using stereo visual odometry.

12. The method of claim 1, wherein an element of a distance matrix is ​​determined as a minimum cosine distance between a flattened overlapping block of the similar view query matrix and the relative view query matrix and the reference matrix.

13. The method of claim 12, wherein the cosine distance is determined by the equation 1-(a·b) / (||a||||b||).

14. The method of claim 13, wherein a diagonal line of the distance matrix including minimum values ​​indicating matches has a slope magnitude approximately equal to + / -1.

15. A system comprising a computer programmed to perform the method of any one of claims 1 to 14.

Images