Camera equipment rotation detection method and device, electronic equipment and storage medium

Abstract

The invention provides a camera equipment rotation detection method and device, electronic equipment and a storage medium, and relates to the technical field of camera shooting, and the method comprises the steps: determining the lane information of a first lane in a current image shot by the camera equipment; for each preset bit image in a preset bit image set corresponding to the camera equipment, determining a difference metric value of the first lane and a first reference lane based on lane information of the first lane and lane information of the first reference lane in the preset bit images; the preset bit image is determined based on lane information of a second reference lane in at least two candidate images shot by the camera device; and under the condition that all the difference metric values are greater than a first preset value, determining that the camera equipment rotates. Whether the camera equipment rotates or not can be determined based on the difference metric value between the lane information, so that the accuracy of rotation detection of the camera equipment is improved.

Description

Technical Field

[0001] The present invention relates to the field of camera technology, and in particular to a camera device rotation detection method, device, electronic equipment and storage medium. Background Art

[0002] Cameras used for monitoring in high-speed traffic scenes are crucial to traffic safety and management. Most of the cameras in high-speed traffic scenes have upgraded their intelligent functions. In order to ensure the accuracy and stability of intelligent function recognition, it is necessary to accurately detect whether the camera equipment is rotating. In high-speed traffic scenes, due to the large volume of traffic, high speed, and fast changes in image information, it brings many challenges to identifying whether the camera equipment is rotating.

[0003] In the related art, a key point detection algorithm is usually used to extract key points in two frames of images taken by a camera device. The key point may be a vehicle, and changes in the distribution of key points in the two frames of images are compared to determine whether the camera device has rotated.

[0004] However, in high-speed traffic scenes, the picture changes frequently. When there are many dynamic targets (vehicles) in the scene or the moving speed is fast, the difference between the two frames of images is also very large when the camera device is not rotating. If the rotation of the camera device is determined based on the changes in the distribution of key points of the two frames of images, the accuracy of the camera device rotation detection will be reduced. Summary of the invention

[0005] In view of the problems existing in the prior art, the embodiments of the present invention provide a method and device for detecting rotation of a camera device, an electronic device and a storage medium.

[0006] The present invention provides a method for detecting rotation of a camera device, comprising:

[0007] Determining lane information of a first lane in a current image captured by the camera device;

[0008] For each pre-position image in the pre-position image set corresponding to the camera device, determine a difference measure value between the first lane and the first reference lane based on lane information of the first lane and lane information of a first reference lane in the pre-position image; the pre-position image is determined based on lane information of a second reference lane in at least two candidate images taken by the camera device;

[0009] When all the difference measurement values ​​are greater than a first preset value, it is determined that the camera device has rotated.

[0010] According to a method for detecting rotation of a camera device provided by the present invention, the method further comprises:

[0011] In the case where there is a difference metric value less than the first preset value among all the difference metric values, determining a first relative offset between the current image and each of the pre-position images based on lane reference lines in a lane reference line set corresponding to each of the pre-position images and a lane line corresponding to the first lane in the current image; the lane reference line is determined based on the variance value of each line segment in the lane line corresponding to the first reference lane in the pre-position image;

[0012] Determining a target relative offset based on each of the first relative offsets;

[0013] When the target relative offset is greater than a second preset value, it is determined that the camera device rotates.

[0014] According to a camera device rotation detection method provided by the present invention, the first relative offset corresponding to the current image and each of the preset position images is determined based on the lane reference lines in the lane reference line set corresponding to each of the preset position images and the lane line corresponding to the first lane in the current image, including:

[0015] Segmenting a lane line corresponding to the first lane in the current image into at least two first line segments;

[0016] For each of the first line segments, determining a second relative offset between the first line segment and the lane reference line based on a first position of the first line segment and a second position on the lane reference line corresponding to the first position;

[0017] For each of the preset position images, based on each of the second relative offsets corresponding to the preset position images, a first relative offset between the current image and the preset position image is determined.

[0018] According to a method for detecting rotation of a camera device provided by the present invention, the method further comprises:

[0019] Determining lane information of a second lane in a target image captured by the camera device, and a third relative offset between the target image and each of the preset position images;

[0020] For each of the pre-positioned images, determining a similarity between the target image and the pre-positioned image based on lane information of the second lane, lane information of the first reference lane in the pre-positioned image, and a third relative offset between the target image and the pre-positioned image;

[0021] When there is a similarity greater than a third preset value among all the similarities, the target image is added to the preset position image set.

[0022] According to a method for detecting rotation of a camera device provided by the present invention, the method further comprises:

[0023] The preset position image corresponding to the smallest similarity among all the similarities is deleted from the preset position image set.

[0024] According to a camera device rotation detection method provided by the present invention, the lane information of the second reference lane includes the lane area and / or the lane direction; the method further includes:

[0025] For each of the candidate images, determining a first preset number of target reference lanes based on the order of lane areas of each of the second reference lanes in the candidate images from large to small;

[0026] For each of the target reference lanes, determining a weight of the target reference lane based on a lane area and / or a lane direction of the target reference lane;

[0027] Determining a first second preset number of target candidate images based on the descending order of the weights of the target reference lanes corresponding to all the candidate images;

[0028] The intersection of all the target candidate images corresponding to each of the target reference lanes is determined to obtain the preset position image set.

[0029] According to a method for detecting rotation of a camera device provided by the present invention, the method further comprises:

[0030] For each of the preset position images, dividing a lane line corresponding to a first reference lane in the preset position image into at least two second line segments;

[0031] determining a variance value of the first position of each of the second line segments;

[0032] Determine a first third preset number of target variance values ​​based on the order of the variance values ​​corresponding to the preset position images from small to large;

[0033] The second line segments corresponding to the target variance values ​​are all determined as lane reference lines in the lane reference line set corresponding to the preset position image.

[0034] The present invention also provides a camera device rotation detection device, comprising:

[0035] A first determining unit, configured to determine lane information of a first lane in a current image captured by the camera device;

[0036] a second determining unit, configured to determine, for each pre-positioned image in the pre-positioned image set corresponding to the camera device, a difference measure value between the first lane and the first reference lane based on lane information of the first lane and lane information of a first reference lane in the pre-positioned image; the pre-positioned image is determined based on lane information of a second reference lane in at least two candidate images taken by the camera device;

[0037] A third determining unit is configured to determine that the camera device has rotated when all the difference measurement values ​​are greater than a first preset value.

[0038] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned methods for detecting rotation of a camera device is implemented.

[0039] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the rotation detection method of the camera device as described in any one of the above is implemented.

[0040] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned methods for detecting rotation of a camera device.

[0041] The camera rotation detection method, device, electronic device and storage medium provided by the present invention pre-set a preset position image set corresponding to the camera based on lane information in at least two candidate images taken by the camera. When determining the lane information of the first lane in the current image taken by the camera, the difference measurement value between the first lane and the first reference lane is determined based on the lane information of the first lane and the lane information of the first reference lane in the preset position image, that is, the difference measurement value between the first lane and each first reference lane is obtained. When all the difference measurement values ​​are greater than the first preset value, it means that there is a difference between the lane in the current image and the lane in the preset position image, so that it can be determined that the camera has rotated. It can be seen that the present invention predetermines a preset image set corresponding to a camera device based on static lane information, calculates a difference measurement value between the lane information of a current image taken by the camera device and the lane information of each preset image in the preset image set, and determines whether the camera device has rotated through the difference measurement value. Since the lane information is a static target, in a high-speed traffic scene, when the camera device does not rotate, the difference in lane information between two frames of images is small, so whether the camera device has rotated can be determined based on the difference measurement value between the lane information, thereby improving the accuracy of the camera device rotation detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 This is one of the flow charts of the method for detecting rotation of a camera device provided by an embodiment of the present invention;

[0044] Figure 2 This is a second flow chart of a method for detecting rotation of a camera device provided by an embodiment of the present invention;

[0045] Figure 3 This is a third flow chart of a method for detecting rotation of a camera device provided by an embodiment of the present invention;

[0046] Figure 4 This is a fourth flow chart of the method for detecting rotation of a camera device provided by an embodiment of the present invention;

[0047] Figure 5 This is a fifth flow chart of the method for detecting rotation of a camera device provided by an embodiment of the present invention;

[0048] Figure 6 is a schematic structural diagram of a rotation detection device for a camera device provided by an embodiment of the present invention;

[0049] Figure 7 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] Combine the following Figure 1-Figure 5 The camera rotation detection method of the present invention is described. The execution subject of the camera rotation detection method can be a camera with intelligent functions, or a computer, server, cloud server or server cluster connected to the camera. The method can be applied in high-speed traffic scenes where the camera is installed, or other ordinary road scenes with lane lines.

[0052] Figure 1FIG. 1 is a flow chart of a method for detecting rotation of a camera device provided by an embodiment of the present invention. Figure 1 As shown, the camera device rotation detection method includes the following steps:

[0053] Step 101: Determine lane information of a first lane in a current image captured by the camera device.

[0054] The camera device is deployed in a captured road scene, the road is divided into multiple lanes, and the first lane is any lane in the current image.

[0055] For example, when it is necessary to detect whether the camera device has rotated, the current image captured by the camera device is obtained, and the current image includes each first lane on the captured road. The first lanes in the current image can be segmented based on the lane segmentation algorithm of deep learning to obtain each first lane. For each first lane, the lane area and lane direction of the first lane are calculated, and the lane area and / or lane direction are determined as the lane information of the first lane. The lane area can be calculated based on the following formula (1), and the lane direction can be calculated based on the following formula (2).

[0056] A i =∫∫D(x,y)dxdy (1)

[0057]

[0058] Among them, A i represents the lane area of ​​the first lane of the i-th lane, D i represents the lane direction of the i-th first lane, D(x,y) represents the binary image of the i-th first lane, represents the directional gradient of the i-th first lane.

[0059] It should be noted that the lane information may also be acquired by using a conventional lane segmentation method, which is not limited in the present invention.

[0060] Step 102: for each pre-position image in the pre-position image set corresponding to the camera device, determine a difference measurement value between the first lane and the first reference lane based on the lane information of the first lane and the lane information of the first reference lane in the pre-position image; the pre-position image is determined based on the lane information of the second reference lane in at least two candidate images taken by the camera device.

[0061] For example, a set of preset images corresponding to the camera device is set in advance based on lane information in at least two candidate images captured by the camera device. For each preset image in the set of preset images, when determining the lane area of ​​each first lane in the current image, the lane areas of each first lane are sorted from large to small to obtain a first sorting result, and the lane areas of each first reference lane in the preset image are sorted from large to small to obtain a second sorting result. The lane information of the first lane with the largest lane area in the first sorting result is compared with the lane information of the first reference lane with the largest lane area in the second sorting result, that is, based on the lane information of the first lane with the largest lane area and the lane information of the first reference lane with the largest lane area, the difference measurement value between the first lane with the largest lane area and the first reference lane with the largest lane area is determined. According to this method, the lane information of the first lane with the second largest lane area in the first sorting result is continuously compared with the lane information of the first reference lane with the second largest lane area in the second sorting result until the lanes in the first sorting result and the lanes in the second sorting result are traversed.

[0062] Specifically, the difference measurement value between the first lane and the first reference lane can be determined based on the following formula (3).

[0063]

[0064] Where L(j) represents the difference between the first lane in the current image and the first reference lane in the j-th preset image, ω represents the weight parameter, k represents the number of lanes in the j-th preset image, and A refj represents the lane area of ​​the first reference lane in the jth preset position image, D refj Indicates the lane direction of the first reference lane in the j-th preset position image.

[0065] It should be noted that, when the number of first lanes in the current image is different from the number of first reference lanes in the preset position image, it indicates that the camera device has rotated, and there is no need to determine the difference measurement value between the first lane and the first reference lane.

[0066] Step 103: When all the difference measurement values ​​are greater than a first preset value, determine that the camera device has rotated.

[0067] The specific value of the first preset value can be set based on demand, and the present invention does not limit this.

[0068] For example, when all the difference metric values ​​between the current image and each preset position image are obtained, all the difference metric values ​​are compared with the first preset value. When all the difference metric values ​​are greater than the first preset value, it means that the difference in lane information between the current image and the preset position images in the preset position image set is large, thereby determining that the camera device has undergone a large-scale rotation, and realizing coarse-grained judgment based on lane information on whether the camera device has rotated. The coarse-grained judgment reduces the performance consumption of the device executing the camera device rotation detection method.

[0069] The camera device rotation detection method provided by the present invention pre-sets a preset position image set corresponding to the camera device based on lane information in at least two candidate images taken by the camera device. When determining the lane information of the first lane in the current image taken by the camera device, the difference measurement value between the first lane and the first reference lane is determined based on the lane information of the first lane and the lane information of the first reference lane in the preset position image, that is, the difference measurement value between the first lane and each first reference lane is obtained. When all the difference measurement values ​​are greater than the first preset value, it means that there is a difference between the lane in the current image and the lane in the preset position image, so that it can be determined that the camera device has rotated. It can be seen that the present invention predetermines a preset image set corresponding to a camera device based on static lane information, calculates a difference measurement value between the lane information of a current image taken by the camera device and the lane information of each preset image in the preset image set, and determines whether the camera device has rotated based on the difference measurement value. Since the lane information is a static target, in a high-speed traffic scene, when the camera device does not rotate, the difference in lane information between two frames of images is small, so whether the camera device has rotated can be determined based on the difference measurement value between the lane information, thereby improving the accuracy of the camera device rotation detection; in addition, the present invention can perform rotation detection only when it is necessary to detect whether the camera device has rotated, reducing unnecessary calculations and improving the operating efficiency of the camera device rotation detection method; moreover, the camera device rotation detection method of the present invention does not rely on any specific environment and equipment except the camera device or the server, has strong versatility, and can be widely used in high-speed traffic scenes, or other ordinary road scenes with lane lines.

[0070] In one embodiment, Figure 2 FIG. 2 is a flow chart of a method for detecting rotation of a camera device provided by an embodiment of the present invention. Figure 2 As shown, the camera device rotation detection method also includes the following steps:

[0071] Step 104: When there is a difference metric value less than the first preset value among all the difference metric values, determine a first relative offset between the current image and each of the preset-position images based on the lane reference lines in the lane reference line set corresponding to each of the preset-position images and the lane line corresponding to the first lane in the current image; the lane reference line is determined based on the variance value of each line segment in the lane line corresponding to the first reference lane in the preset-position image.

[0072] For example, when there is a difference metric value less than the first preset value among all the difference metric values, it means that there is no difference or a small difference in lane information between the current image and the preset position images in the preset position image set. At this time, it is necessary to use the lane reference lines in the lane reference line set corresponding to each preset position image to make a fine-grained judgment on whether the camera device has rotated. That is, when there is a difference metric value less than the first preset value among all the difference metric values, based on the lane reference lines in the lane reference line set corresponding to the preset position image and the lane line corresponding to the first lane in the current image, the first relative offset corresponding to the current image and the preset position image is determined. According to the same method, the current image and each preset position image are calculated pairwise to obtain the first relative offset corresponding to the current image and each preset position image.

[0073] Step 105: Determine a target relative offset based on each of the first relative offsets.

[0074] For example, when the first relative offsets corresponding to the current image and each preset position image are obtained, the smallest first relative offset among all the first relative offsets may be determined as the target relative offset.

[0075] Step 106: When the target relative offset is greater than a second preset value, determine that the camera device has rotated.

[0076] For example, when the target relative offset is obtained, the target relative offset is compared with the second preset value. When the target relative offset is greater than the second preset value, it means that there is a slight difference between the current image and the preset position image in the preset position image set, thereby indicating that the camera device has undergone a small range of rotation; when the target relative offset is less than or equal to the second preset value, it means that there is no slight difference between the current image and the preset position image in the preset position image set, thereby indicating that the camera device has not undergone any rotation. Specifically, the following formula (4) can be used to determine whether the camera device has undergone a small range of rotation.

[0077] min (|σ x1 |,|σ x2 |,…|σ xm |)> T x (4)

[0078] Among them, σ xm represents the first relative offset between the current image and the mth preset position image in the x-axis direction, min(|σ x1 |,|σ x2 |,…|σ xm |) represents the target relative offset, T x It represents the second preset value. The specific value of the second preset value can be set based on demand, and the present invention is not limited to this.

[0079] In this embodiment, when there is a difference metric value less than the first preset value among all the difference metric values, a fine-grained judgment can be made on whether the camera device has rotated based on the lane reference lines in the lane reference line set corresponding to the preset position image and the lane lines corresponding to the first lane in the current image, thereby further improving the accuracy of the camera device rotation detection.

[0080] In one embodiment, in the above step 104, based on the lane reference lines in the lane reference line set corresponding to each of the preset position images and the lane line corresponding to the first lane in the current image, determining the first relative offset corresponding to the current image and each of the preset position images can be specifically implemented in the following manner:

[0081] The lane line corresponding to the first lane in the current image is divided into at least two first line segments; for each of the first line segments, based on a first position of the first line segment and a second position on the lane reference line corresponding to the first position, a second relative offset between the first line segment and the lane reference line is determined; for each of the preset position images, based on each of the second relative offsets corresponding to the preset position image, a first relative offset between the current image and the preset position image is determined.

[0082] Among them, the area between the two lane lines is a first lane, the first position and the second position can be center positions, or the first position and the second position can be positions of preset proportions, for example, the preset proportion is 1 / 3, etc. The first position and the second position are taken as center positions as examples below.

[0083] For example, by using the interception method, a straight line parallel to the x-axis is used to intercept the lane line of the first lane in the current image into j first line segments. One first lane corresponds to two lane lines, and each lane line is intercepted into j second line segments, where j is an integer greater than or equal to 2. Similarly, the lane reference lines in the lane reference line set are also obtained by intercepting the straight line parallel to the x-axis. Because the first line segments intercepted based on the straight line parallel to the x-axis have the same y coordinate of the center position of any line segment, for each first line segment, the x coordinate of the center position of the first line segment and the x coordinate of the center position of the lane reference line in the lane reference line set are calculated. The difference in coordinates is called the second relative offset between the first line segment and the lane reference line, that is, the x-coordinate offset. According to the same method, the difference between the x-coordinate of the center position of the first line segment and the x-coordinate of the center position of each lane reference line in the lane reference line set can be calculated, that is, the x-coordinate offsets corresponding to the current image and a preset position image are obtained, and the minimum x-coordinate offset among the x-coordinate offsets is determined as the first relative offset corresponding to the current image and the preset position image; for each preset position image in the preset position image set, the first relative offset corresponding to the current image and each preset position image can be obtained.

[0084] In this embodiment, the lane line is divided into at least two line segments, and the first relative offset corresponding to the current image and the preset position image is determined based on the second relative offset between the first line segment of the first lane in the current image and the lane reference line in the lane reference line set corresponding to the preset position image, thereby achieving a finer-grained division, making the determined first relative offset more accurate, and further improving the accuracy of determining whether the camera device has rotated based on the first relative offset.

[0085] In one embodiment, Figure 3 FIG. 3 is a flow chart of a method for detecting rotation of a camera device provided by an embodiment of the present invention. Figure 3 As shown, after step 103, the camera device rotation detection method further includes the following steps:

[0086] Step 107: Determine lane information of the second lane in the target image captured by the camera device, and a third relative offset between the target image and each of the preset position images.

[0087] Among them, since the high-speed traffic scene is outdoors and the environment is changing all the time, the preset position image set is the cornerstone for judging whether the camera device is rotating. Therefore, updating the preset position image set is a key step. The second lane is any lane in the target image.

[0088] For example, a new image taken by a camera device, i.e., a target image, is obtained, and based on the same method as the above step 101, lane information of the second lane in the target image is determined, the lane information of the second lane includes the lane area and / or lane direction, and then based on the same method as the above method for determining the first relative offset corresponding to the current image and each preset position image, a third relative offset between the target image and each preset position image is determined.

[0089] Step 108: For each of the pre-positioned images, determine the similarity between the target image and the pre-positioned image based on the lane information of the second lane, the lane information of the first reference lane in the pre-positioned image, and a third relative offset between the target image and the pre-positioned image.

[0090] For example, an update strategy of a pre-position image set based on similarity can be used. A similarity function S is defined, which receives two inputs, one input is a new image I taken by a camera device, that is, a target image I, and the other input is a pre-position image P in the pre-position image set, and returns the similarity between the target image I and the pre-position image P. The similarity function S makes a judgment based on the lane area, lane direction, and the third relative offset of the pre-position image, and specifically determines the similarity between the target image and each pre-position image based on the following formula (5).

[0091]

[0092] Where S(I,P) represents the similarity between the target image I and the preset position image P, α represents the weight coefficient corresponding to the lane area, β represents the weight coefficient corresponding to the lane direction, γ represents the weight coefficient corresponding to the third relative offset, and A I represents the lane area of ​​the second lane in the target image I, D I represents the lane direction of the second lane in the target image I, X I -X P represents the third relative offset between the target image I and the preset position image P, A p represents the lane area of ​​the first reference lane in the preset position image P, D P Indicates the lane direction of the first reference lane in the preset position image P.

[0093] It should be noted that α, β and γ in the above formula (5) can be adjusted according to needs, and the present invention does not limit this.

[0094] Step 109: When there is a similarity greater than a third preset value among all the similarities, add the target image to the preset position image set.

[0095] The third preset value may be set based on demand, and the present invention does not limit this.

[0096] For example, when the similarities between the target image and each preset position image are obtained, a search is performed among all similarities to determine whether there is a similarity greater than a third preset value. When a similarity greater than the third preset value is found among all similarities, it indicates that the target image is similar to a preset position image in the preset position image set. Because the environment, such as light, changes over time, the newly captured target image reflects more accurate lane information than the preset position image corresponding to the minimum similarity. Therefore, the newly captured target image is added to the preset position image set, thereby implementing an update of the preset position image set.

[0097] Furthermore, the preset position image corresponding to the minimum similarity among all the similarities is deleted from the preset position image set.

[0098] For example, S(I, P) calculated based on the target image I and each preset-position image P is screened, and the preset-position image with the smallest similarity to the target image I is removed from the preset-position image set, thereby ensuring the reliability of each preset-position image in the preset-position image set as much as possible.

[0099] It should be noted that, considering that the lane information reflected by the preset position image with a longer storage time has a lower similarity with the current lane information, the preset position image with the longest storage time in the preset position image set can also be deleted to update the preset position image set, and the present invention is not limited to this.

[0100] It should be noted that the present invention adopts a method for dynamically maintaining the preset position image set, and may also adopt a method for updating the preset position image set at fixed time intervals, which is not limited in the present invention.

[0101] In this embodiment, while adding a new preset position image to the preset position image set, the preset position image corresponding to the minimum similarity is eliminated, thereby ensuring the stability and real-time performance of the preset position image set, and improving the consistency between the preset position image and the current road environment, which is beneficial to improving the accuracy of the camera device rotation detection. The present invention dynamically maintains the preset position image set, including adding and eliminating preset position images, so that the present invention has strong robustness to some minor changes in lane information caused by weather changes and lighting changes in the images collected by the camera device, as well as the noise of the segmentation results; in addition, by updating the preset position image set in real time, the present invention can also take into account the slow changes of the external environment, such as weather changes, lighting changes, etc., in real time, so that the camera device rotation detection method of the present invention has stronger adaptability and real-time performance.

[0102] In one embodiment, Figure 4 FIG. 4 is a flow chart of a method for detecting rotation of a camera device provided in an embodiment of the present invention. Figure 4 As shown, the lane information of the second reference lane includes the lane area and / or the lane direction; before step 101, the camera device rotation detection method further includes the following steps:

[0103] Step 110: for each of the candidate images, determine a first preset number of target reference lanes based on the order of lane areas of the second reference lanes in the candidate images from large to small.

[0104] Among them, the lane segmentation results of different images in the same scene may be different, and the reliability of using the lane information of a single frame image as a pre-position image is low. To address this problem, the present invention uses a method for optimizing pre-position images to aggregate the lane information of at least two candidate images to obtain a relatively reliable set of pre-position images to prevent misjudgment of camera device rotation.

[0105] For example, for each candidate image, the lane area of ​​each second reference lane in the candidate image is calculated based on the above formula (1), and the lane areas of each second reference lane in the candidate image are arranged in order from large to small, and the second reference lanes corresponding to the first preset number of lane areas are all determined as target reference lanes. For example, if the first preset number is 2, the second reference lanes corresponding to the first two lane areas are both determined as target reference lanes.

[0106] Step 111: For each target reference lane, determine the weight of the target reference lane based on the lane area and / or lane direction of the target reference lane.

[0107] For example, taking the lane information of the target reference lane including the lane area and lane direction as an example, a weight function W(i) is introduced to calculate the weights of the first two target reference lanes with the largest area in each candidate image by combining the variance of the lane area and the lane direction. Specifically, the weight of the target reference lane can be determined based on the following formula (6).

[0108]

[0109] Where W(i) represents the weight of the i-th target reference lane, α, β and γ represent the parameters for adjusting the weight, and μ A represents the mean lane area of ​​all second reference lanes in the candidate image, μ D represents the mean of the lane directions of all second reference lanes in the candidate image, σ A represents the variance of lane areas of all second reference lanes in the candidate image, σ D Represents the variance of lane directions of all second reference lanes in the candidate image.

[0110] Step 112: Determine a first second preset number of target candidate images based on the descending order of the weights of the target reference lanes corresponding to all the candidate images.

[0111] For example, when the weights of the target reference lanes corresponding to the candidate images are obtained, for the target reference lanes with the largest lane area in all the candidate images, the weights of the target reference lanes with the largest lane area corresponding to all the candidate images are sorted from large to small, and the candidate images corresponding to the first second preset number of weights are all determined as target candidate images; for example, the first preset number is 2 and the second preset number is 4, then the number of target reference lanes is 2, namely target reference lane 1 with the largest lane area and target reference lane 2 with the second largest lane area, for the target reference lane 1 with the largest lane area, the candidate images corresponding to the first 4 weights in all the candidate images are determined as target candidate images to obtain set 1, for the target reference lane 2 with the second largest lane area, the candidate images corresponding to the first 4 weights in all the candidate images are determined as target candidate images 2 to obtain set 2, for example, set 1 includes image 1, image 2, image 3 and image 4, and set 2 includes image 3, image 4, image 5 and image 6.

[0112] It should be noted that the above is a comparison method based on weight size to determine the target candidate image. A comparison method based on Euclidean distance or Manhattan distance may also be used to determine the target candidate image, and the present invention does not limit this.

[0113] Step 113: determine the intersection of all the target candidate images corresponding to each of the target reference lanes to obtain the preset position image set.

[0114] For example, the intersection of set 1 and set 2 is determined as the preset position image set. For example, set 1 includes image 1, image 2, image 3 and image 4, and set 2 includes image 3, image 4, image 5 and image 6, then image 3 and image 4 are determined as elements in the preset position image set.

[0115] It should be noted that the preset position image set may also be determined using an image input by a user, or based on a preset position image set by default in a device, and the present invention does not limit this.

[0116] In this embodiment, the second reference lane with a larger lane area is determined as the target reference lane, and the weight of the target reference lane is determined based on the lane area and / or lane direction, and then the candidate image corresponding to the larger weight is selected as the pre-position image in the pre-position image set, so that the pre-position image has a lane with a larger area and a larger weight, so that based on the pre-position images in the pre-position image set, it is easier to detect the difference between the lanes, thereby improving the accuracy of the rotation detection of the camera equipment.

[0117] In one embodiment, Figure 5 FIG. 5 is a flow chart of a method for detecting rotation of a camera device provided by an embodiment of the present invention. Figure 5 As shown, before step 101, the camera device rotation detection method further includes the following steps:

[0118] Step 114: For each of the preset position images, segment the lane line corresponding to the first reference lane in the preset position image into at least two second line segments.

[0119] For example, for each preset position image in the preset position image set, the lane line of the first reference lane in the preset position image is intercepted into j second line segments by using a straight line parallel to the x-axis through the interception method. One first reference lane corresponds to two lane lines, and each lane line is intercepted into j second line segments, which can be expressed as L ik (k=1,2,…j),L ik Represents the kth second line segment in the lane line of the first reference lane in the i-th preset position image.

[0120] Step 115: Determine the variance value of the first position of each of the second line segments.

[0121] The first position may be a center position or a position of a preset ratio, for example, the preset ratio is 1 / 3, etc. The following is an example in which the first position is taken as the center position.

[0122] For example, for each second line segment in the preset position image, two endpoints X of the second line segment are calculated. ik1 and X ik2 The mean of ik , calculate the variance Var(X) of the midpoint of the kth second line segment in all pre-position images in the pre-position image set k ), specifically, the variance value of the first position of the k-th second line segment can be calculated based on the following formula (7), for example, the variance value of the midpoint of the k-th second line segment is calculated.

[0123]

[0124] Among them, μ Xk represents the X of the kth second line segment in all pre-positioned images in the pre-positioned image set ik , m represents the set of preset position images.

[0125] Step 116: Determine the first third preset number of target variance values ​​based on the order of the variance values ​​corresponding to the preset position images from small to large.

[0126] For example, the variance values ​​corresponding to a preset position image are sorted from small to large, and the first third preset number of variance values ​​are determined as target variance values; for example, if the third preset number is 5, the first 5 variance values ​​are determined as target variance values.

[0127] Step 117: Determine the second line segments corresponding to the target variance values ​​as lane reference lines in the lane reference line set corresponding to the preset position image.

[0128] For example, when each target variance value corresponding to a preset position image is obtained, the second line segments corresponding to the target variance values ​​among the j second line segments are determined as lane reference lines in the lane reference line set corresponding to the preset position image. The lane reference line set corresponding to each preset position image can be determined according to the above method, and the present invention will not be repeated here; specifically, the following formula (8) can be used to express the lane reference line set R corresponding to the preset position image.

[0129] R = argsort(Var(X k ))[:g],k=1,2,…j (8)

[0130] Where g represents the number of lane reference lines in the lane reference line set R, argsort is a function that transforms Var(X k ) array is sorted according to the variance size, and then the index value is returned, that is, the first third preset number of second line segments with the smallest variance. In this way, a set of lane reference lines is obtained, ensuring that the obtained lane reference lines are relatively stable reference objects in the image captured by the camera device. The lane reference lines of each subsequent frame are obtained by intercepting the same straight line parallel to the x-axis and the lane, and the present invention will not be repeated here.

[0131] In this embodiment, lane reference lines are optimized using information such as mean and variance, so that a more reliable set of lane reference lines can be obtained, which is more reliable than setting a simple preset position image, and comprehensively considers three factors: lane area, lane direction and lane reference lines, so that the lane information can be described more comprehensively and accurately.

[0132] To sum up, the camera device rotation detection method of the present invention effectively improves the performance of camera device rotation detection in high-speed traffic scenarios, including accuracy, robustness, real-time, high efficiency and versatility, which is of great significance for improving the working efficiency of camera devices in high-speed traffic scenarios.

[0133] The following is a description of the camera rotation detection device provided by the present invention. The camera rotation detection device described below and the camera rotation detection method described above can be referred to each other.

[0134] Figure 6is a schematic diagram of the structure of a camera rotation detection device provided by an embodiment of the present invention. Figure 6 As shown, the camera device rotation detection device 600 includes a first determination unit 601, a second determination unit 602 and a third determination unit 603; wherein:

[0135] A first determining unit 601 is used to determine lane information of a first lane in a current image captured by the camera device;

[0136] a second determining unit 602, configured to determine, for each pre-positioned image in the pre-positioned image set corresponding to the camera device, a difference measure value between the first lane and the first reference lane based on lane information of the first lane and lane information of a first reference lane in the pre-positioned image; the pre-positioned image is determined based on lane information of a second reference lane in at least two candidate images taken by the camera device;

[0137] The third determining unit 603 is configured to determine that the camera device has rotated when all the difference measurement values ​​are greater than a first preset value.

[0138] The camera rotation detection device provided by the present invention pre-sets a preset position image set corresponding to the camera based on lane information in at least two candidate images taken by the camera. When determining the lane information of the first lane in the current image taken by the camera, the difference measurement value between the first lane and the first reference lane is determined based on the lane information of the first lane and the lane information of the first reference lane in the preset position image, that is, the difference measurement value between the first lane and each first reference lane is obtained. When all the difference measurement values ​​are greater than the first preset value, it means that there is a difference between the lane in the current image and the lane in the preset position image, so that it can be determined that the camera has rotated. It can be seen that the present invention predetermines a preset image set corresponding to a camera device based on static lane information, calculates a difference measurement value between the lane information of a current image taken by the camera device and the lane information of each preset image in the preset image set, and determines whether the camera device has rotated through the difference measurement value. Since the lane information is a static target, in a high-speed traffic scene, when the camera device does not rotate, the difference in lane information between two frames of images is small, so whether the camera device has rotated can be determined based on the difference measurement value between the lane information, thereby improving the accuracy of the camera device rotation detection.

[0139] Based on any of the above embodiments, the camera device rotation detection device 600 further includes:

[0140] a fourth determination unit, configured to determine, when there is a difference metric value less than the first preset value among all the difference metric values, a first relative offset corresponding to the current image and each of the preset images based on lane reference lines in a lane reference line set corresponding to each of the preset images and a lane line corresponding to the first lane in the current image; the lane reference line is determined based on a variance value of each line segment in the lane line corresponding to the first reference lane in the preset image;

[0141] a fifth determining unit, configured to determine a target relative offset based on each of the first relative offsets;

[0142] The sixth determining unit is used to determine that the camera device has rotated when the target relative offset is greater than a second preset value.

[0143] Based on any of the foregoing embodiments, the fourth determining unit is specifically configured to:

[0144] Segmenting a lane line corresponding to the first lane in the current image into at least two first line segments;

[0145] For each of the first line segments, determining a second relative offset between the first line segment and the lane reference line based on a first position of the first line segment and a second position on the lane reference line corresponding to the first position;

[0146] For each of the preset position images, based on each of the second relative offsets corresponding to the preset position images, a first relative offset between the current image and the preset position image is determined.

[0147] Based on any of the above embodiments, the camera device rotation detection device 600 further includes:

[0148] a seventh determining unit, configured to determine lane information of a second lane in the target image captured by the camera device, and a third relative offset between the target image and each of the preset position images;

[0149] an eighth determining unit, configured to determine, for each of the pre-positioned images, a similarity between the target image and the pre-positioned image based on lane information of the second lane, lane information of the first reference lane in the pre-positioned image, and a third relative offset between the target image and the pre-positioned image;

[0150] An adding unit is used to add the target image to the preset position image set when there is a similarity greater than a third preset value among all the similarities.

[0151] Based on any of the above embodiments, the camera device rotation detection device 600 further includes:

[0152] The deleting unit is used to delete the preset position image corresponding to the minimum similarity among all the similarities from the preset position image set.

[0153] Based on any of the above embodiments, the lane information of the second reference lane includes lane area and / or lane direction; the camera device rotation detection device 600 also includes:

[0154] a ninth determining unit, configured to determine, for each of the candidate images, a first preset number of target reference lanes based on the order of lane areas of each of the second reference lanes in the candidate images from large to small;

[0155] a tenth determining unit, configured to determine, for each of the target reference lanes, a weight of the target reference lane based on a lane area and / or a lane direction of the target reference lane;

[0156] an eleventh determining unit, configured to determine a first second preset number of target candidate images based on the descending order of the weights of the target reference lanes corresponding to all the candidate images;

[0157] The twelfth determining unit is used to determine the intersection of all the target candidate images corresponding to each of the target reference lanes to obtain the preset position image set.

[0158] Based on any of the above embodiments, the camera device rotation detection device 600 further includes:

[0159] a segmentation unit, configured to segment, for each of the preset position images, a lane line corresponding to a first reference lane in the preset position image into at least two second line segments;

[0160] a thirteenth determining unit, configured to determine a variance value of the first position of each of the second line segments;

[0161] A fourteenth determining unit, configured to determine a first third preset number of target variance values ​​based on the order of the variance values ​​corresponding to the preset position images from small to large;

[0162] The fifteenth determining unit is used to determine the second line segments corresponding to the target variance values ​​as lane reference lines in the lane reference line set corresponding to the preset position image.

[0163] Figure 7 is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention, such as Figure 7As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the camera device rotation detection method, the method including: determining the lane information of the first lane in the current image captured by the camera device;

[0164] For each pre-position image in the pre-position image set corresponding to the camera device, determine a difference measure value between the first lane and the first reference lane based on lane information of the first lane and lane information of a first reference lane in the pre-position image; the pre-position image is determined based on lane information of a second reference lane in at least two candidate images taken by the camera device;

[0165] When all the difference measurement values ​​are greater than a first preset value, it is determined that the camera device has rotated.

[0166] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0167] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the camera device rotation detection method provided by the above methods, the method includes: determining the lane information of the first lane in the current image captured by the camera device;

[0168] For each pre-position image in the pre-position image set corresponding to the camera device, determine a difference measure value between the first lane and the first reference lane based on lane information of the first lane and lane information of a first reference lane in the pre-position image; the pre-position image is determined based on lane information of a second reference lane in at least two candidate images taken by the camera device;

[0169] When all the difference measurement values ​​are greater than a first preset value, it is determined that the camera device has rotated.

[0170] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, the computer program being implemented when executed by a processor to perform the camera device rotation detection method provided by the above methods, the method comprising: determining lane information of a first lane in a current image captured by the camera device;

[0171] For each pre-position image in the pre-position image set corresponding to the camera device, determine a difference measure value between the first lane and the first reference lane based on lane information of the first lane and lane information of a first reference lane in the pre-position image; the pre-position image is determined based on lane information of a second reference lane in at least two candidate images taken by the camera device;

[0172] When all the difference measurement values ​​are greater than a first preset value, it is determined that the camera device has rotated.

[0173] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0174] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0175] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting rotation of a camera device, characterized in that: include: Determining lane information of a first lane in a current image captured by the camera device; For each pre-position image in the pre-position image set corresponding to the camera device, determine a difference measure value between the first lane and the first reference lane based on lane information of the first lane and lane information of a first reference lane in the pre-position image; the pre-position image is determined based on lane information of a second reference lane in at least two candidate images taken by the camera device; When all the difference measurement values ​​are greater than a first preset value, it is determined that the camera device has rotated.

2. The method for detecting rotation of a camera device according to claim 1, characterized in that: The method further comprises: In the case where there is a difference metric value less than the first preset value among all the difference metric values, determining a first relative offset between the current image and each of the pre-position images based on lane reference lines in a lane reference line set corresponding to each of the pre-position images and a lane line corresponding to the first lane in the current image; the lane reference line is determined based on the variance value of each line segment in the lane line corresponding to the first reference lane in the pre-position image; Determining a target relative offset based on each of the first relative offsets; When the target relative offset is greater than a second preset value, it is determined that the camera device rotates.

3. The method for detecting rotation of a camera device according to claim 2, characterized in that: The determining, based on the lane reference lines in the lane reference line set corresponding to each of the preset position images and the lane line corresponding to the first lane in the current image, a first relative offset corresponding to the current image and each of the preset position images, comprises: Segmenting a lane line corresponding to the first lane in the current image into at least two first line segments; For each of the first line segments, determining a second relative offset between the first line segment and the lane reference line based on a first position of the first line segment and a second position on the lane reference line corresponding to the first position; For each of the preset position images, based on each of the second relative offsets corresponding to the preset position images, a first relative offset between the current image and the preset position image is determined.

4. The method for detecting rotation of a camera device according to claim 1, characterized in that: The method further comprises: Determining lane information of a second lane in a target image captured by the camera device, and a third relative offset between the target image and each of the preset position images; For each of the pre-positioned images, determining a similarity between the target image and the pre-positioned image based on lane information of the second lane, lane information of the first reference lane in the pre-positioned image, and a third relative offset between the target image and the pre-positioned image; When there is a similarity greater than a third preset value among all the similarities, the target image is added to the preset position image set.

5. The method for detecting rotation of a camera device according to claim 4, characterized in that: The method further comprises: The preset position image corresponding to the smallest similarity among all the similarities is deleted from the preset position image set.

6. The method for detecting rotation of a camera device according to any one of claims 1 to 5, characterized in that: The lane information of the second reference lane includes lane area and / or lane direction; the method further includes: For each of the candidate images, determining a first preset number of target reference lanes based on the order of lane areas of each of the second reference lanes in the candidate images from large to small; For each of the target reference lanes, determining a weight of the target reference lane based on a lane area and / or a lane direction of the target reference lane; Determining a first second preset number of target candidate images based on the descending order of the weights of the target reference lanes corresponding to all the candidate images; The intersection of all the target candidate images corresponding to each of the target reference lanes is determined to obtain the preset position image set.

7. The method for detecting rotation of a camera device according to any one of claims 3 to 5, characterized in that: The method further comprises: For each of the preset position images, dividing a lane line corresponding to a first reference lane in the preset position image into at least two second line segments; determining a variance value of the first position of each of the second line segments; Determine a first third preset number of target variance values ​​based on the order of the variance values ​​corresponding to the preset position images from small to large; The second line segments corresponding to the target variance values ​​are all determined as lane reference lines in the lane reference line set corresponding to the preset position image.

8. A camera rotation detection device, characterized in that: include: A first determining unit, configured to determine lane information of a first lane in a current image captured by the camera device; a second determining unit, configured to determine, for each pre-positioned image in the pre-positioned image set corresponding to the camera device, a difference measure value between the first lane and the first reference lane based on lane information of the first lane and lane information of a first reference lane in the pre-positioned image; the pre-positioned image is determined based on lane information of a second reference lane in at least two candidate images taken by the camera device; A third determining unit is configured to determine that the camera device has rotated when all the difference measurement values ​​are greater than a first preset value.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for detecting rotation of an imaging device according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting rotation of an imaging device according to any one of claims 1 to 7 is implemented.

Images