Image-based positioning method, position relationship judgment method and device

Abstract

This patent discloses an image-based positioning method, a position relationship judgment method and a device thereof, which belongs to the field of computer vision and is used to restore the real position information of the target in the image according to the image, eliminate the information loss effect of the 3D to 2D conversion process when the image is captured, and on this basis, construct an accurate position relationship between the targets. Through this accurate position relationship, a series of problems related to recognition caused by inaccurate target position judgment can be effectively solved. The main technical scheme of the present invention is: step S1, obtain an image taken by a camera and identify the key points of the target in the image; step S2, restore the real position information of the key points according to the parameters of the camera; step S3, combine the real position information of the key points and statistical data to obtain the real size information of the target and obtain the real position information.

Description

Technical Field

[0001] The present invention relates to the field of intelligent video monitoring, and in particular to an image-based positioning method, a position relationship judgment method and a device thereof. Background Art

[0002] With the continuous advancement of science and technology, image-based automation systems have begun to emerge in many fields. For example, in the field of transportation, many intelligent transportation systems have been derived, among which video-based intelligent transportation systems have attracted widespread attention. In the field of intelligent warehousing, intelligent management systems based on image recognition have also become a future development direction.

[0003] There are many solutions based on video or image to achieve intelligent management in the market. For example, the parking system based on video can capture the vehicles entering and leaving the parking spaces on both sides of the road through video to achieve effective management of parking spaces. The warehouse management system based on video can judge the storage status of goods through images.

[0004] However, the intelligent system based on video surveillance is limited by 2D image processing. The positional relationship in the image does not actually reflect the real positional relationship, resulting in unclear positioning of objects in the image, and therefore the positional relationship between objects in the image cannot be accurately determined. In the parking monitoring scenario, this problem is reflected in the fact that due to the limitation of 2D images, it is impossible to accurately judge the relationship between the vehicle and the parking space, resulting in complaints caused by wrong capture, and it is impossible to effectively judge the parking situation of the vehicle, such as abnormal parking, cross-space parking, diagonal parking, side parking, etc., resulting in chaos in parking space management. In the warehousing scenario, this problem is reflected in the fact that due to the limitation of 2D images, it is impossible to accurately judge the placement and placement relationship of the goods, which will affect the system's automated judgment process. Therefore, how to eliminate the information loss during 3D to 2D conversion and accurately judge the real positional relationship of objects based on the image has become an urgent problem to be solved. Summary of the invention

[0005] The present invention provides an image-based positioning method, a position relationship judgment method and a device thereof, which are used to solve the problem of low accuracy when judging the real position relationship through images.

[0006] An embodiment of the present invention provides an image-based positioning method, the method comprising:

[0007] Step S1, obtaining an image taken by a camera and identifying key points of the target in the image;

[0008] Step S2, restoring the real position information of the key point according to the parameters of the camera;

[0009] Step S3, combining the real position information of the key points with the statistical data, obtaining the real size information of the target and obtaining the real position information;

[0010] Optionally, the target includes at least one of the following objects: a vehicle, a traffic participant, a traffic facility, a road marking, and a reference object within a scene.

[0011] Optionally, the key points are points necessary for measuring the size of the target and are easily recognized by humans or computers, including at least one of the following key points: the endpoint of the target contour line, the intersection of the target contour line, the center point of a regular figure on the target, and a protruding structure on the target.

[0012] Optionally, when the target is a parking space:

[0013] In step S1, the key points include two end points of the parking space line on the side facing the camera, and the coordinates of the key points in the image can be obtained by algorithm recognition or manual input;

[0014] In step S3, the real size information of the target can be obtained by algorithm recognition or manual input.

[0015] Optionally, when the target is a vehicle:

[0016] In step S1, the key points include: the center point of the wheel, the tangent point of the wheel and the ground, the upper and lower end points of the pillars on both sides of the rear windshield, the two end points of the rear bumper, the upper and lower end points of the pillars on both sides of the front windshield, the end points of the lighting devices on both sides of the front of the vehicle, and the two end points of the front bumper;

[0017] In step S3, the real size information of the target includes: length information, width information and height information of the vehicle;

[0018] Optionally, the length information of the vehicle is the product of the distance between two wheels on the same side of the vehicle and a proportional coefficient, and the proportional coefficient is obtained from statistical data of the ratio of the distance between two wheels on the same side to the vehicle length information.

[0019] Optionally, the width information of the vehicle is the sum of the distance between the two end points of the front or rear bumper of the vehicle and the horizontal distance between the two end points of the bumper and the tangent point of the wheel and the ground, or the product of the distance between the two end points of the front or rear bumper of the vehicle and a proportional coefficient, where the proportional coefficient is the ratio of the distance between the two end points of the front or rear bumper of the vehicle to the vehicle width information, and is obtained through statistical data.

[0020] Optionally, the height information of the vehicle is the vertical distance between the upper end points of the pillars on both sides of the front and rear windshield of the vehicle and the line connecting the tangent points of the wheels and the ground.

[0021] The present invention provides a method for determining a position relationship based on an image, characterized in that the method comprises:

[0022] Step P1, obtaining the real position information of the target according to the above-mentioned image-based positioning method, where the number of the targets is at least two;

[0023] Step P2, determining the overlap relationship between the targets, and judging the position relationship between the targets in combination with the usage scenario.

[0024] Optionally, when the target is a vehicle and a parking space, and the usage scenario is a parking scenario, step P2 includes:

[0025] Step P201, removing the height information from the real position information of the target, and obtaining the position information of the parking space rectangular frame and the vehicle rectangular frame;

[0026] Step P202, calculating the overlap rate between the parking space rectangular frame and the vehicle rectangular frame;

[0027] Step P203, judging the parking situation of the vehicle according to the overlap rate.

[0028] An embodiment of the present invention provides a device for determining a position relationship based on an image, the device comprising:

[0029] An image acquisition module is used to acquire images taken by a camera;

[0030] Recognition module, used to identify the key points of the target in the image;

[0031] A conversion module, used for converting the image position information of the key points into real position information;

[0032] A positioning module, used to obtain the real size information of the target and obtain the real position information according to the real position information of the key points;

[0033] The judgment module is used to judge the real position relationship between the targets according to the real position information of the targets.

[0034] An embodiment of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the above-mentioned image positioning methods and any one of the above-mentioned image-based position relationship determination methods when executing the computer program.

[0035] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image positioning method described in any one of the above items and the image-based position relationship judgment method described in any one of the above items are implemented.

[0036] The present invention provides an image-based positioning method, a position relationship judgment method and its device, a computer device and a storage medium. First, in step S1, an image taken by a camera is obtained to identify key points of a target in the image; in step S2, the real position information of the key points is restored according to the parameters of the camera; in step S3, the real size information of the target is obtained by combining the real position information of the key points with statistical data, and the real position information is obtained;

[0037] The image-based positioning method provided by the present invention can restore the real position information of the target in the image according to the image, and eliminate the influence of information loss in the 3D to 2D conversion process when the image is captured. The position relationship judgment method constructs an accurate position relationship between targets on the basis of the results obtained by the image-based positioning method. Through this accurate position relationship, a series of problems caused by inaccurate target position judgment can be effectively solved. For example, in the field of intelligent transportation, the present invention can be applied to a parking system based on video monitoring, which can improve the accuracy of judging the position relationship between vehicles and parking spaces, and effectively solve the problem of identifying passing vehicles as parked vehicles. In the field of intelligent warehousing, the present invention can be applied to a video-based warehousing management system, which can improve the accuracy of judging the position relationship between target goods and the goods storage area, and effectively solve the problem of low recognition rate of goods storage and transfer due to irregular placement. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0039] Figure 1 is a general flow chart of an image-based positioning method in one embodiment of the present invention;

[0040] Figure 2 is a schematic diagram of the camera installation position in one embodiment of the present invention;

[0041] Figure 3 Schematic diagram of three national standard parking spaces in one embodiment of the present invention

[0042] Figure 4 Schematic diagram of the key point position when the left rear side of the vehicle faces the camera in one embodiment of the present invention

[0043] Figure 5 Schematic diagram of the key point position when the right front side of the vehicle faces the camera in one embodiment of the present invention

[0044] Figure 6 is a block diagram of an image-based position relationship determination device in one embodiment of the present invention;

[0045] Figure 7 It is a device diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] In one embodiment of the present invention, an image-based positioning method is provided, such as Figure 1 As shown, the method includes:

[0048] Step S1, obtaining an image taken by a camera and identifying key points of the target in the image;

[0049] The image is a picture taken by a camera or a frame in a video taken by a video camera.

[0050] When the positioning method is applied to the field of intelligent transportation, the target includes at least one of the following objects: common objects in the transportation field such as vehicles, traffic participants, traffic facilities, road markings, reference objects in the scene, etc.

[0051] When the positioning method is applied to the field of intelligent warehousing, the targets include common objects in the warehousing field such as goods, shelves, goods transport devices, and warehouse markings.

[0052] It is foreseeable that when the positioning is applied to other applicable scenarios that require image-based positioning and judgment of positional relationships, technical personnel in this field can apply the method without any creative work, so the target also includes all objects that can be positioned based on images.

[0053] The key points are points that are necessary for measuring the size of the target and are easily recognized by humans or computers, including at least one of the following key points: the endpoints of the target contour lines, the intersections of the target contour lines, the center points of regular figures on the target, and protruding structures on the target.

[0054] The selection of key points is adapted to the appearance of the target, but under the guidance of the present method, key points can be determined for all possible targets without the need for creative work by those skilled in the art.

[0055] Step S2, restoring the real position information of the key point according to the parameters of the camera;

[0056] The camera parameters are obtained after camera calibration, and specifically include the intrinsic parameters and extrinsic parameters of the camera. The intrinsic parameters of the camera are parameters related to the camera's own characteristics, such as the camera's focal length, pixel size, etc.; the extrinsic parameters of the camera are parameters in the world coordinate system, such as the camera's position, rotation direction, etc. By calibrating the camera parameters using existing technology, the correspondence between the image coordinates and the real world coordinates can be obtained. Through this correspondence, the coordinates of the key points in the world coordinate system, that is, the real position information of the key points, can be restored.

[0057] Step S3, combining the real position information of the key points with the statistical data, obtaining the real size information of the target and obtaining the real position information;

[0058] For a planar target, the real size information is the length or width of the target; when the planar target is other regular shapes, its size information can also be expressed as information specific to the regular shape, such as diameter, circumference, etc.; the real position information includes the position information of the actual contour line or abstract contour line of the planar target in the world coordinate system, which can be expressed as the equation of the overall contour line, the equation of a partial contour line, the coordinates of several points on the contour line, etc.

[0059] For example, when the plane target is a rectangular ground line, its size information is the length and width of the rectangle, and its real position information can be expressed as the equation of the rectangle in the world coordinate system, or as the equations of the two sides of the rectangle in the world coordinate system.

[0060] For a target presented in the form of a line, the real size information includes information such as the length of the line and the curvature of the line; the real position information can be expressed as the equation of the line in the world coordinate system or the coordinates of several points on the line, etc.

[0061] For example, when the target is a straight line segment on the ground, the real size information includes the length of the line segment; the real position information can be expressed by the equation of the line in the world coordinate system, or by the coordinates of the two end points of the line segment in the world coordinate system.

[0062] For a 3D target, the real size information includes the size information of the target's actual contour line or the size information of the abstract contour line; the abstract contour line is the contour line of a regular 3D figure obtained by abstracting the 3D target. The real position information can be expressed by the equation of the actual 3D target or the abstracted 3D target in the world coordinate system, or by necessary planes and / or straight lines and / or endpoints.

[0063] For example, when the target is a rectangular carton, its true size information is the size information of the actual contour line of the rectangular carton, namely, length, width and height; the true position information can be represented by three planes of the rectangle, or by three straight lines of the length, width and height of the rectangle.

[0064] The real size information of the target is determined by the above method, and the representation of the boundary of the target in the world coordinate system, that is, the real position information of the target, can be obtained based on the real size information.

[0065] Embodiment 1:

[0066] When the image-based positioning method is applied to the field of intelligent transportation and the target is a parking space, the method includes:

[0067] Step S1, obtaining an image taken by a camera, and obtaining the coordinates of the starting point and the end point of the parking space line in the image.

[0068] The camera is a common gun-ball linkage shooting device in the field of intelligent transportation, in which the gun monitors the status of the parking space in real time, and the ball camera captures the license plate information of the corresponding parking space according to the instructions of the gun. Figure 2 described.

[0069] The image captured by the camera is a video frame, the parking space line is the parking space line close to the lens, and the starting point and end point coordinates of the parking space line can be added when the parking space is manually configured, or can be automatically identified and saved through a recognition algorithm when the parking space is empty. The recognition algorithm is a prior art.

[0070] Step S2, restoring the real position information of the key point according to the parameters of the camera;

[0071] The camera parameters are obtained by calibrating the camera. For the camera calibration method, please refer to another invention application "A parking scene camera calibration method, device, computer equipment and storage medium" submitted by the applicant on the same day. After obtaining the camera parameters, the mapping relationship between the image coordinates and the world coordinates and the coordinates of the vanishing points in the three directions of xyz can be obtained.

[0072] According to the mapping relationship, the starting and ending coordinates of the parking space line are converted into coordinates in the world coordinate system.

[0073] Step S3, combining the real position information and statistical data of the key points, obtaining the real size information of the target and obtaining the real position information:

[0074] The size of parking spaces is based on the national standards for parking spaces. Small parking spaces are 6 meters long and 2.5 meters wide, suitable for small vehicles. Parking space arrangements can be divided into three types: parallel, inclined and vertical. Figure 3 As shown:

[0075] Balanced parking space: The standard length is 6 meters and the width is 2.5 meters.

[0076] Inclined parking space: The inclined length reaches 6 meters, the width is 2.8 meters, and the vertical distance between the two inclined lines should be maintained at the standard of 2.5 meters.

[0077] Vertical parking space: length is greater than or equal to 5 meters, usually set to 6 meters in length and 2.5 meters in width. Generally, 2.5x5.3m is the best standard parking space size.

[0078] According to the parking space type determined manually or automatically according to the algorithm when the parking space is pre-configured, the other two coordinate points of the parking space are calculated in the world coordinate system to obtain the position information of the parking space in the world coordinate system.

[0079] The real position information of the parking space is expressed in the form of the coordinates of a certain coordinate point of the parking space combined with the length and width information of the parking space (x, y, width, length).

[0080] Embodiment 2:

[0081] When the image-based positioning method is applied to the field of intelligent transportation, when the target is a vehicle, affected by the camera installation position, the vehicle in the image will have four postures in the three national standard parking spaces (see Figure 4 ): 1. The left rear side of the vehicle faces the lens; 2. The left front side of the vehicle faces the lens; 3. The right rear side of the vehicle faces the lens; 4. The right front side of the vehicle faces the lens. Since the camera needs to be installed to ensure that all vehicles are facing the lens at a certain angle, there is basically no absolute situation where the front or rear of the vehicle faces the lens.

[0082] In this embodiment, the image-based positioning method is described by taking the left rear side of the vehicle facing the camera as an example. The judgment methods under other viewing angles are similar to those in this embodiment. Those skilled in the art can perform image-based positioning of vehicles under other three viewing angles according to the description of this embodiment without creative work, so it will not be repeated here.

[0083] When the left rear side of the vehicle faces the camera, the image-based positioning method includes:

[0084] Step S1, obtaining an image taken by a camera and identifying key points of the target in the image;

[0085] The key points include: wheel center points (4), wheel and ground tangent points (4), upper and lower endpoints of the rear windshield pillars (4), two end points of the rear bumper (2), upper and lower endpoints of the front windshield pillars (4), endpoints of the lighting devices on both sides of the front (2), and two end points of the front bumper (2), a total of 22 key points. For specific distribution and numbering, see Figure 4 and Figure 5 .

[0086] Affected by the above-mentioned viewing angle, the number of key points detected for a target is not fixed. In this embodiment, the left rear side of the vehicle faces the camera, and the key points identified are (see Figure 4 ) The center points of the two wheels on the left side of the vehicle (16, 18), the tangent points of the left wheels of the vehicle and the ground (20, 22), the two end points of the rear bumper (13, 14), and the upper end point of the left pillar of the rear windshield (9, 11), a total of 7 key points.

[0087] This solution uses YOLACT to achieve the target instance segmentation task and identifies the above key points through instance segmentation.

[0088] Step S2, restoring the real position information of the key point according to the parameters of the camera;

[0089] The camera parameters are the result of camera calibration. For the camera calibration method, please refer to another invention application "A parking scene camera calibration method, device, computer equipment and storage medium" filed by the applicant on the same day. After obtaining the camera parameters according to the above method, the mapping relationship between the image coordinates and the world coordinates and the coordinates of the vanishing points in the three directions of xyz can be obtained.

[0090] According to the mapping relationship, the coordinates of the seven key points in step S1 are converted into coordinates in the world coordinate system.

[0091] Step S3, combining the real position information of the key points with the statistical data, obtaining the real size information of the target and obtaining the real position information;

[0092] The real size information of the target includes: length information, width information and height information of the vehicle;

[0093] The length information of the vehicle is the product of the distance between the two wheels on the same side of the vehicle and the proportionality coefficient. The proportionality coefficient is obtained from the statistical data of the ratio of the distance between the wheels on the same side to the vehicle length information. The specific calculation method is as follows:

[0094] Known wheel center coordinates , since the actual height of the wheel center is the same, the z direction is approximately the same and can be ignored. The length L of the vehicle can be expressed as the distance between the wheel center points multiplied by the proportional coefficient α (different models correspond to different proportional coefficients, and the proportional coefficient here is obtained based on statistical information). The formula is as follows:

[0095]

[0096] The width information of the vehicle is the sum of the distance between the two end points of the front or rear bumper of the vehicle and the horizontal distance between the two end points of the bumper and the tangent point of the wheel and the ground, or the product of the distance between the two end points of the front or rear bumper of the vehicle and the proportional coefficient. The proportional coefficient is obtained from the statistical data of the ratio of the distance between the two end points of the front or rear bumper of the vehicle to the vehicle width information. The specific calculation method is as follows:

[0097] Get the coordinates of the two end points of the rear bumper in the world coordinate system There are two methods to calculate vehicle width information. One is:

[0098]

[0099] Wherein, W is the width of the vehicle, and b is the distance from the left end point P3 of the rear bumper facing the camera to the side plane of the vehicle.

[0100] The method for calculating b is: calculate the equation of the line connecting the two tangent points of the left wheel of the vehicle and the ground in the world coordinate system, project P3 onto the road plane, and calculate the distance from the projection point of P3 to the tangent point line. This distance is the distance b from P3 to the side plane of the vehicle.

[0101] The second width calculation method is similar to the method for calculating the vehicle length information in this embodiment, and the vehicle width information is obtained by multiplying the distance between the two end points of the bumper by the proportionality coefficient obtained from statistical data.

[0102] The height information of the vehicle is the vertical distance between the upper end points of the pillars on both sides of the front and rear windshields of the vehicle and the line connecting the tangent points of the wheels and the ground. The specific method for obtaining the height information is as follows:

[0103] The first step is to find the line m1 connecting the two tangent points between the left wheel of the vehicle and the ground in the image;

[0104] Step 2: In the image, connect the upper end point of the left pillar of the rear windshield with the vertical vanishing point to obtain a straight line m2 passing through the upper end point of the left pillar of the rear windshield and perpendicular to the ground.

[0105] The third step is to calculate the coordinates of the intersection of m1 and m2 in the image, and transform them into the coordinates P5 (x5, y5, 0) in the world coordinate system according to the corresponding relationship. Since the intersection is on the ground, the value in the z direction should be 0;

[0106] Step 4: The upper end point of the left pillar of the rear windshield of the vehicle is located in the z direction of the intersection, and its coordinate in the world coordinate system should be P6 (x5, y5, z), where z is the height information of the vehicle.

[0107] The length, width and height information of the target are obtained by the above method. According to the above key point coordinates, straight line equations and vehicle size information, the position of the cube abstracted by the vehicle in the world coordinate system is obtained.

[0108] In one embodiment of the present invention, a method for determining a positional relationship based on an image is provided, the method comprising:

[0109] Step P1, according to the above-mentioned image-based positioning method, obtain the real position information of the target, the number of which is at least two. The real position information of the target, that is, the position of the target in the world coordinate system, can be obtained by the image-based positioning method.

[0110] Step P2, determining the overlap relationship between the targets, and judging the position relationship between the targets in combination with the usage scenario. The method for determining the overlap relationship between the targets is:

[0111] The real position information of the targets in step P1 is processed respectively, one dimension of the three-dimensional information is removed, and the overlap rate of multiple targets in a certain direction is calculated;

[0112] According to the overlap rate of multiple targets in a specific direction and combined with the prior knowledge of the actual application scenario, the positional relationship between the targets is judged.

[0113] Example 3

[0114] In this embodiment, the target is a vehicle and a parking space, and the usage scenario is a parking scenario. The image-based position relationship determination method includes:

[0115] Step P1, according to the above-mentioned image-based positioning method, obtain the real position information of the target. The specific implementation method is referred to Example 1 and Example 2.

[0116] Step P2, determining the overlap relationship between the targets, and judging the position relationship between the targets in combination with the usage scenario, specifically includes:

[0117] Step P201, remove the height information in the real position information of the target, and obtain the position information of the parking space rectangular frame and the vehicle rectangular frame. The rectangle is represented by the starting point and the width and height (x, y, width, height)

[0118] Step P202, calculate the overlap rate between the parking space rectangular frame and the vehicle rectangular frame, and use the intersection of the parking space frame A and the target frame B to divide the union to calculate the overlap rate OverLap:

[0119]

[0120] Step P203, judging the parking situation of the vehicle according to the overlap rate.

[0121] If the projection area is inside the parking space or the overlap rate is higher than a certain threshold, it is considered normal parking;

[0122] When the projection area is outside the parking space or the overlap rate is below a certain threshold, it can be inferred that the vehicle is an interfering vehicle and should not be captured or the illegal parking capture should be initiated;

[0123] If the overlap rate between the projection area and the parking space is within a certain range, it can be determined that it is abnormal parking. If the projection matrix spans two parking spaces, it can be determined that the target parking space is straddling parking. If the vehicle does not occupy two parking spaces, the vehicle is diagonally parked.

[0124] The above thresholds can be adjusted according to the actual road conditions and relevant factors, and are not specifically limited in this embodiment.

[0125] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0126] Embodiment 4:

[0127] In the field of intelligent transportation, illegal parking capture generally sets up illegal parking areas, and then determines whether the vehicle is in the illegal parking area to achieve illegal parking capture. The accuracy of position judgment directly affects the subsequent illegal parking penalty. Here, the vehicle and the illegal parking area can be used as targets to determine whether the parking position of the vehicle falls into the illegal parking area. The illegal parking area can be a virtual line stored in advance. Through the image-based position relationship judgment method, it is possible to accurately determine whether there is a situation of illegal parking of the vehicle. The specific implementation method is similar to Example 3.

[0128] Embodiment 5:

[0129] In the field of intelligent transportation, the traffic flow statistics on the road are generally achieved by setting an area of ​​interest on the road, and then counting the targets passing through the area of ​​interest. Vehicles and areas of interest can also be used as targets to achieve effective judgment of vehicle targets and areas of interest. The area of ​​interest can be a virtual line stored in advance, or it can be an existing line or other marker actually existing on the road. Through the image-based position relationship judgment method, the position relationship between the vehicle and the area of ​​interest can be accurately judged, thereby achieving accurate counting. The specific implementation method is similar to Example 3, but compared with Example 3, this embodiment has an additional counting step to perform traffic flow statistics.

[0130] Embodiment 6:

[0131] In the field of intelligent transportation, this solution can also be used for vehicle speed calculation. By constructing a 3D position relationship, the vehicle can be transformed from the image coordinate system to the overhead plane coordinate system, and then the accurate position relationship between the vehicle position and the road plane can be calculated in the overhead plane. The position relationship can be used in combination with time to calculate the vehicle speed. Through the image-based position relationship judgment method, the change in the position relationship between the vehicle and the road plane can be accurately judged by comparing two frames, and the vehicle speed can be accurately calculated by combining the shooting interval of the two frames.

[0132] Those skilled in the art can reasonably predict that when the implementation methods given in the specification are applied to different targets, the methods can achieve the technical effects based on accurate image positioning and positional relationship judgment, and are equivalent alternatives or obvious variations of the embodiments of the present invention.

[0133] In one embodiment, a device for determining a positional relationship based on an image is provided. The device for determining a positional relationship based on an image corresponds to the device for determining a positional relationship based on an image and the method for determining a positional relationship based on an image in the above-mentioned embodiment. The device includes (eg Figure 6 shown):

[0134] 10. An image acquisition module, used to acquire images taken by a camera;

[0135] The camera is a common image capturing device in the field of intelligent transportation, such as a gun-type camera, a spherical camera or a gun-ball linked camera.

[0136] 20. Recognition module, used to identify the key points of the target in the image:

[0137] The image is a photo taken by a camera or a frame in a video taken by a video camera.

[0138] When the recognition module is applied to the field of intelligent transportation, the target includes at least one of the following objects: common objects in the transportation field such as transportation tools, traffic participants, traffic facilities, road markings, reference objects in the scene, etc.

[0139] When the recognition module is applied to the field of intelligent warehousing, the targets include common objects in the warehousing field such as goods, shelves, goods transport devices, warehouse markings, etc.

[0140] The key points are points that are necessary for measuring the size of the target and are easily recognized by humans or computers, including at least one of the following key points: the endpoints of the target contour lines, the intersections of the target contour lines, the center points of regular figures on the target, and protruding structures on the target.

[0141] 30, a conversion module, for converting the image position information of the key point into the real position information:

[0142] Restoring the real position information of the key point is completed based on the parameters of the camera. The camera parameters are obtained after camera calibration, and specifically include the intrinsic parameters and extrinsic parameters of the camera. The intrinsic parameters of the camera are parameters related to the camera's own characteristics, such as the camera's focal length, pixel size, etc.; the extrinsic parameters of the camera are parameters in the world coordinate system, such as the camera's position, rotation direction, etc. By calibrating the camera parameters through existing technology, the correspondence between the image coordinates and the real world coordinates can be obtained. Through this correspondence, the coordinates of the key point in the world coordinate system, that is, the real position information of the key point, can be restored.

[0143] 40, a positioning module, used to obtain the real size information of the target and the real position information according to the real position information of the key point;

[0144] The positioning module comprises a target size information calculation unit (401) and a target real position information storage unit (402).

[0145] The target size information calculation unit (401) is used to calculate the size information of the target according to the real position information of the key points. For a plane target, the real size information is the length or width of the target; when the plane target is other regular shapes, its size information can also be expressed as information specific to the regular shape, such as diameter, circumference, etc.; the real position information includes the position information of the actual contour line or abstract contour line of the plane target in the world coordinate system, which can be expressed as the equation of the whole contour line, the equation of a part of the contour line, the coordinates of several points on the contour line, etc. For example, when the plane target is a rectangular ground line, its size information is the length and width of the rectangle, and its real position information can be expressed as the equation of the rectangle in the world coordinate system, or as the equation of the two sides of the rectangle in the world coordinate system.

[0146] For a target presented in the form of a line, the real size information includes information such as the length and curvature of the line; the real position information can be expressed as the equation of the line in the world coordinate system or the coordinates of several points on the line, etc. For example, when the target is a straight line segment on the ground, the real size information includes the length of the line segment; the real position information can be expressed as the equation of the line in the world coordinate system, or as the coordinates of the two end points of the line segment in the world coordinate system.

[0147] For a three-dimensional target, the real size information includes the size information of the actual contour line of the target or the size information of the abstract contour line; the abstract contour line is the contour line of a regular three-dimensional figure obtained by abstracting the three-dimensional target. The real position information can be expressed by the equation of the actual three-dimensional target or the abstracted three-dimensional target in the world coordinate system, or by necessary planes and / or straight lines and / or endpoints. For example, when the target is a rectangular carton, its real size information is the size information of the actual contour line of the rectangular carton, namely, the length, width and height; the real position information can be expressed by the three planes of the rectangle, or by the three straight lines of the length, width and height of the rectangle.

[0148] The target real position information storage unit (402) is used to store the target real position information obtained based on the target size information: the representation of the target boundary in the world coordinate system can be obtained based on the real size information, that is, the target real position information.

[0149] 50, a judgment module, used to judge the real position relationship between the targets according to the real position information of the targets;

[0150] The real position information of the target is obtained from the positioning module, and the number of the targets is at least two. The real position information of the target is the position of the target in the world coordinate system.

[0151] The overlap relationship between the objects is determined, and the position relationship between the objects is determined based on the usage scenario.

[0152] The method for determining the coincidence relationship of the targets is:

[0153] The real position information of the targets is processed respectively, one dimension in the three-dimensional information is removed, and the overlap rate of multiple targets in a certain direction is calculated; based on the overlap rate of multiple targets in a specific direction and combined with prior knowledge of actual application scenarios, the positional relationship between the targets is determined.

[0154] For the specific limitations of the image-based positional relationship judgment device, please refer to the limitations of the image-based positioning method and the image-based positional relationship judgment method in the above text, which will not be repeated here. Each module in the above-mentioned image-based positional relationship judgment device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0155] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an image-based positioning method and an image-based position relationship judgment method are implemented.

[0156] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0157] Step S1, obtaining an image taken by a camera and identifying key points of the target in the image;

[0158] Step S2, restoring the real position information of the key point according to the parameters of the camera;

[0159] Step S3, combining the real position information of the key points with the statistical data, obtaining the real size information of the target and obtaining the real position information;

[0160] Optionally, the target includes at least one of the following objects: a vehicle, a traffic participant, a traffic facility, a road marking, and a reference object within a scene.

[0161] Optionally, the key points are points necessary for measuring the size of the target and are easily recognized by humans or computers, including at least one of the following key points: the endpoint of the target contour line, the intersection of the target contour line, the center point of a regular figure on the target, and a protruding structure on the target.

[0162] Optionally, when the target is a parking space:

[0163] In step S1, the key points include two end points of the parking space line on the side facing the camera, and the coordinates of the key points in the image can be obtained by algorithm recognition or manual input;

[0164] In step S3, the real size information of the target can be obtained by algorithm recognition or manual input.

[0165] Optionally, when the target is a vehicle:

[0166] In step S1, the key points include: the center point of the wheel, the tangent point of the wheel and the ground, the upper and lower end points of the pillars on both sides of the rear windshield, the two end points of the rear bumper, the upper and lower end points of the pillars on both sides of the front windshield, the end points of the lighting devices on both sides of the front of the vehicle, and the two end points of the front bumper;

[0167] In step S3, the real size information of the target includes: length information, width information and height information of the vehicle;

[0168] Optionally, the length information of the vehicle is the product of the distance between two wheels on the same side of the vehicle and a proportional coefficient, and the proportional coefficient is obtained from statistical data of the ratio of the distance between two wheels on the same side to the vehicle length information.

[0169] Optionally, the width information of the vehicle is the sum of the distance between the two end points of the front or rear bumper of the vehicle and the horizontal distance between the two end points of the bumper and the tangent point of the wheel and the ground, or the product of the distance between the two end points of the front or rear bumper of the vehicle and a proportional coefficient, and the proportional coefficient is obtained from statistical data of the ratio of the distance between the two end points of the front or rear bumper of the vehicle to the vehicle width information.

[0170] Optionally, the height information of the vehicle is the vertical distance between the upper end points of the pillars on both sides of the front and rear windshield of the vehicle and the line connecting the tangent points of the wheels and the ground.

[0171] And executing a method for determining a position relationship based on an image, the method comprising:

[0172] Step P1, according to the above-mentioned image-based positioning method, obtain the real position information of the target, and the number of the targets is at least two.

[0173] Step P2, determining the overlap relationship between the targets, and judging the position relationship between the targets in combination with the usage scenario.

[0174] Optionally, when the target is a vehicle and a parking space, and the usage scenario is a parking scenario, step P2 includes:

[0175] Step P201, removing the height information from the real position information of the target, and obtaining the position information of the parking space rectangular frame and the vehicle rectangular frame;

[0176] Step P202, calculating the overlap rate between the parking space rectangular frame and the vehicle rectangular frame;

[0177] Step P203, judging the parking situation of the vehicle according to the overlap rate.

[0178] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0179] Step S1, obtaining an image taken by a camera and identifying key points of the target in the image;

[0180] Step S2, restoring the real position information of the key point according to the parameters of the camera;

[0181] Step S3, combining the real position information of the key points with the statistical data, obtaining the real size information of the target and obtaining the real position information;

[0182] Optionally, the target includes at least one of the following objects: a vehicle, a traffic participant, a traffic facility, a road marking, and a reference object within a scene.

[0183] Optionally, the key points are points necessary for measuring the size of the target and are easily recognized by humans or computers, including at least one of the following key points: the endpoint of the target contour line, the intersection of the target contour line, the center point of a regular figure on the target, and a protruding structure on the target.

[0184] Optionally, when the target is a parking space:

[0185] In step S1, the key points include two end points of the parking space line on the side facing the camera, and the coordinates of the key points in the image can be obtained by algorithm recognition or manual input;

[0186] In step S3, the real size information of the target can be obtained by algorithm recognition or manual input.

[0187] Optionally, when the target is a vehicle:

[0188] In step S1, the key points include: the center point of the wheel, the tangent point of the wheel and the ground, the upper and lower end points of the pillars on both sides of the rear windshield, the two end points of the rear bumper, the upper and lower end points of the pillars on both sides of the front windshield, the end points of the lighting devices on both sides of the front of the vehicle, and the two end points of the front bumper;

[0189] In step S3, the real size information of the target includes: length information, width information and height information of the vehicle;

[0190] Optionally, the length information of the vehicle is the product of the distance between two wheels on the same side of the vehicle and a proportional coefficient, and the proportional coefficient is obtained from statistical data of the ratio of the distance between two wheels on the same side to the vehicle length information.

[0191] Optionally, the width information of the vehicle is the sum of the distance between the two end points of the front or rear bumper of the vehicle and the horizontal distance between the two end points of the bumper and the tangent point of the wheel and the ground, or the product of the distance between the two end points of the front or rear bumper of the vehicle and a proportional coefficient, and the proportional coefficient is obtained from statistical data of the ratio of the distance between the two end points of the front or rear bumper of the vehicle to the vehicle width information.

[0192] Optionally, the height information of the vehicle is the vertical distance between the upper end points of the pillars on both sides of the front and rear windshield of the vehicle and the line connecting the tangent points of the wheels and the ground.

[0193] And executing a method for determining a position relationship based on an image, the method comprising:

[0194] Step P1, according to the above-mentioned image-based positioning method, obtain the real position information of the target, and the number of the targets is at least two.

[0195] Step P2, determining the overlap relationship between the targets, and judging the position relationship between the targets in combination with the usage scenario.

[0196] Optionally, when the target is a vehicle and a parking space, and the usage scenario is a parking scenario, step P2 includes:

[0197] Step P201, removing the height information from the real position information of the target, and obtaining the position information of the parking space rectangular frame and the vehicle rectangular frame;

[0198] Step P202, calculating the overlap rate between the parking space rectangular frame and the vehicle rectangular frame;

[0199] Step P203, judging the parking situation of the vehicle according to the overlap rate.

[0200] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0201] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0202] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. An image-based positioning method, characterized in that: The method comprises: Step S1, obtaining an image captured by a camera, and identifying key points of multiple targets in the image, wherein the multiple targets include multiple vehicles and multiple parking spaces; Step S2, restoring the real position information of the key points of the multiple targets in the same world coordinate system according to the parameters of the camera; Step S3, combining the real position information and statistical data of the key points of the multiple targets in the same world coordinate system, obtaining the real size information of the target, and obtaining the representation of the boundary of the target in the world coordinate system according to the real size information of the target as the real position information of the target; The statistical data includes statistical data of multiple parking space types; the real position information and statistical data of the key points of the multiple targets in the same world coordinate system are combined to obtain the real size information of the target, and the representation of the boundary of the target in the world coordinate system is obtained according to the real size information of the target, and the real position information of the target includes: In response to the target being a parking space, determining a parking space type; Determining actual size information of the parking space according to statistical data corresponding to the parking space type; and According to the real position information of the key points of the parking space and the real size information of the parking space, the representation of the boundary of the parking space in the world coordinate system is determined.

2. The image-based positioning method according to claim 1, characterized in that: The target includes at least one of the following objects: a vehicle, a traffic participant, a traffic facility, a road marking, and a reference object in a scene.

3. The image-based positioning method according to claim 1, characterized in that: The key points are points that are necessary for measuring the size of the target and are easily recognized by humans or computers, including at least one of the following key points: the endpoints of the target contour lines, the intersections of the target contour lines, the center points of regular figures on the target, and protruding structures on the target.

4. The image-based positioning method according to claim 1, characterized in that: When the target is a parking space: In step S1, the key points include two end points of the parking space line on the side facing the camera, and the coordinates of the key points in the image are obtained by algorithm recognition or manual input; In step S3, the real size information of the target is obtained by algorithm recognition or manual input.

5. The image-based positioning method according to claim 1, characterized in that: When the target is a vehicle: In step S1, the key points include: the center point of the wheel, the tangent point of the wheel and the ground, the upper and lower end points of the pillars on both sides of the rear windshield, the two end points of the rear bumper, the upper and lower end points of the pillars on both sides of the front windshield, the end points of the lighting devices on both sides of the front of the vehicle, and the two end points of the front bumper; In step S3, the real size information of the target includes: length information, width information and height information of the vehicle.

6. The image-based positioning method according to claim 5, characterized in that: The length information of the vehicle is the product of the distance between two wheels on the same side of the vehicle and a proportional coefficient, and the proportional coefficient is obtained from statistical data of the ratio of the distance between two wheels on the same side to the vehicle length information.

7. The image-based positioning method according to claim 5, characterized in that: The width information of the vehicle is the sum of the distance between the two end points of the front or rear bumper of the vehicle and the horizontal distance between the two end points of the bumper and the tangent point of the wheel and the ground, or the product of the distance between the two end points of the front or rear bumper of the vehicle and a proportional coefficient. The proportional coefficient is the ratio of the distance between the two end points of the front or rear bumper of the vehicle to the vehicle width information, which is obtained through statistical data.

8. The image-based positioning method according to claim 5, characterized in that: The height information of the vehicle is the vertical distance between the upper end points of the pillars on both sides of the front and rear windshields of the vehicle and the connection points between the wheels and the ground.

9. A method for determining a positional relationship based on an image, characterized in that: The method comprises: Step P1, according to the image-based positioning method of claim 1, obtaining real position information of the target, wherein the number of the targets is at least two; Step P2, determining the overlap relationship between the targets, and judging the position relationship between the targets in combination with the usage scenario.

10. The method for determining positional relationship based on an image according to claim 9, characterized in that: When the target is a vehicle and a parking space, and the usage scenario is a parking scenario, step P2 includes: Step P201, removing the height information from the real position information of the target, and obtaining the position information of the parking space rectangular frame and the vehicle rectangular frame; Step P202, calculating the overlap rate between the parking space rectangular frame and the vehicle rectangular frame; Step P203, judging the parking situation of the vehicle according to the overlap rate.

11. A device for determining a positional relationship based on an image, characterized in that: The device comprises: An image acquisition module is used to acquire images taken by a camera; A recognition module, used to recognize key points of multiple targets in an image, wherein the multiple targets include multiple vehicles and multiple parking spaces; A conversion module, used for converting the image position information of key points of multiple targets into real position information in the same world coordinate system; A positioning module, used to obtain the real size information of the target according to the real position information and statistical data of the key points of the multiple targets, and obtain the representation of the boundary of the target in the world coordinate system according to the real size information of the target as the real position information of the target; A judgment module, used to judge the real position relationship between targets according to the real position information of the targets; Wherein, the statistical data includes statistical data of multiple parking space types; The positioning module is used to determine the type of parking space in response to the target being a parking space; determine the real size information of the parking space based on statistical data corresponding to the parking space type; and determine the representation of the boundary of the parking space in the world coordinate system based on the real position information of the key points of the parking space and the real size information of the parking space.

12. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the image positioning method according to any one of claims 1 to 8 and the image-based position relationship determination method according to any one of claims 9 to 10 are implemented.

13. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image positioning method according to any one of claims 1 to 8 and the image-based position relationship determination method according to any one of claims 9 to 10 are implemented.

Images