Camera position deviation detection method, electronic device and storage medium

Abstract

The embodiments of the present application provide a method, electronic device, and storage medium for detecting camera position offsets. The method includes: obtaining position range information of road-related facilities, where the position range information of the road-related facilities is pre-determined based on first road video data collected by the camera; obtaining second road video data collected by the camera; detecting the position information of active objects based on the second road video data; matching the position information of the active objects with the position range information of the road-related facilities to determine a matching result; generating an offset alarm message when the matching result satisfies an offset condition; and sending the offset alarm message to adjust the camera's shooting angle. By comparing active objects on the road with static road facilities and analyzing whether interactive objects appear at locations where static road facilities should not appear, the method can detect offsets and issue an alarm in a timely manner, while also ensuring the accuracy of matching results based on camera images.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for detecting position offset of a camera, an electronic device, and a storage medium. Background Art

[0002] The camera is prone to deviation during continuous operation (polling) at the preset position, resulting in changes in the captured image and inaccurate subsequent processing results based on the captured image.

[0003] Taking road cameras as an example, when the road video footage does not meet expectations, the event judgment rules based on traffic regulations become invalid, resulting in false alarms. For example, the wrong approaching or going lane leads to a false alarm of wrong-way driving, and the wrong motor vehicle lane leads to pedestrians walking in the fast lane or motor vehicles occupying the emergency lane.

[0004] Therefore, a technical problem that those skilled in the art urgently need to solve is: timely detecting camera deviation. Summary of the Invention

[0005] The embodiment of the present application provides a method for detecting position deviation of a camera, so as to promptly detect deviation of the camera.

[0006] Correspondingly, the embodiment of the present application also provides an electronic device and a storage medium to ensure the implementation and application of the above system.

[0007] In order to solve the above problems, an embodiment of the present application discloses a method for detecting position offset of a camera, the method comprising:

[0008] Acquiring location range information of road-related facilities, where the location range information of the road-related facilities is pre-determined based on first road video data collected by a camera;

[0009] Obtaining second road video data collected by the camera;

[0010] detecting position information of a moving object based on the second road video data;

[0011] Matching the location information of the active object with the location range information of the road-related facilities to determine a matching result;

[0012] When the matching result satisfies the offset condition, generating offset alarm information;

[0013] The deviation alarm information is sent to adjust the shooting angle of the camera.

[0014] Optionally, detecting the location information of the moving object based on the second road video data includes:

[0015] performing target recognition based on each frame of road image in the second road video data to determine at least one target object;

[0016] Tracking the at least one target object in the second road video data to determine at least one moving object;

[0017] Position information of the at least one active object is determined.

[0018] Optionally, matching the location information of the active object with the location range information of the road-related facility to determine a matching result includes:

[0019] comparing the location information of the active object with the location range information of the road-related facility;

[0020] Counting overlapping information of the location information of the active object within the location range information of the road-related facility;

[0021] Generate matching results based on overlapping information.

[0022] Optionally, the types of the active objects include: motorized and non-motorized types, and the road-related facilities include: motorized and non-motorized types;

[0023] The comparing the location information of the active object with the location range information of the road-related facility includes:

[0024] Comparing the location information of the motorized activity object with the location range information of the non-motorized associated road facilities to determine whether the two overlap; and / or,

[0025] Compare the location information of non-motorized activity objects with the location range information of motorized road-related facilities to determine whether the two overlap.

[0026] Optionally, the counting of overlapping information of the location information of the active object appearing within the location range information of the road-related facility includes:

[0027] Counting the number of times that the location information of the active object appears within the location range information of the road-related facility within a set time;

[0028] The overlapping frequency is counted according to the set time and the number of overlapping times.

[0029] Optionally, when the matching result satisfies the offset condition, generating offset alarm information includes:

[0030] Determine whether the overlapping information exceeds the overlapping threshold;

[0031] If the overlapping information exceeds the overlapping threshold, it is determined that the offset condition is met and an offset alarm message is generated.

[0032] Optionally, it also includes: analyzing offset information based on the position information of the overlapping active objects and the position range information of the road-related facilities, the offset information including: offset direction and / or offset angle; and adding the offset information to the offset alarm information.

[0033] Optionally, the method further includes: determining the location range information of the road-related facilities based on the first road video data collected by the camera in advance.

[0034] Optionally, the determining the location range information of the road-related facilities based on the first road video data collected by the camera in advance includes:

[0035] Acquire each frame of road image from the first road video data;

[0036] Performing semantic segmentation on each frame of road image to determine a segmentation result;

[0037] The location range information of the road-related facilities is determined according to the segmentation result.

[0038] Optionally, the determining the location range information of the road-related facilities based on the first road video data collected by the camera in advance includes:

[0039] The first road video data is input into a semantic segmentation model for semantic segmentation, and a segmentation result is output; and location range information of road-related facilities is determined based on the segmentation result.

[0040] Optionally, the road-related facilities include: roads and road facilities; the roads include motor vehicle lanes, non-motor vehicle lanes, and sidewalks; the road facilities include at least one of the following: traffic signs, markings, pedestrian overpasses, pedestrian tunnels, separation facilities, road display screens, lighting equipment, and bus stops; the separation facilities include at least one of the following: guardrails, bollards, green belts, and flower beds.

[0041] The present application discloses a method for detecting position offset of a camera, wherein the method includes:

[0042] Acquiring location range information of a static facility, where the location range information of the static facility is determined based on first video data collected by a camera;

[0043] Acquire second video data captured by the camera;

[0044] detecting position information of a moving object based on the second video data;

[0045] Matching the location information of the active object with the location range information of the static facility to determine a matching result;

[0046] When the matching result satisfies the offset condition, generating offset alarm information;

[0047] The deviation alarm information is sent to adjust the shooting angle of the camera.

[0048] The embodiment of the present application discloses a device for detecting position deviation of a camera, characterized in that the device includes:

[0049] a facility determination module, configured to obtain location range information of road-related facilities, wherein the location range information of the road-related facilities is pre-determined based on first road video data collected by a camera;

[0050] a position detection module, configured to obtain second road video data captured by a camera; and detect position information of a moving object based on the second road video data;

[0051] an offset detection module, configured to match the location information of the moving object with the location range information of the road-related facilities to determine a matching result;

[0052] The alarm module is used to generate an offset alarm message when the matching result meets the offset condition; and send the offset alarm message to adjust the shooting angle of the camera.

[0053] An embodiment of the present application further discloses an electronic device, comprising: a processor; and a memory on which executable code is stored. When the executable code is executed by the processor, the method described in the embodiment of the present application is executed.

[0054] The embodiments of the present application further disclose one or more machine-readable media on which executable codes are stored. When the executable codes are executed by a processor, the method described in the embodiments of the present application is executed.

[0055] Compared with the prior art, the embodiments of the present application have the following advantages:

[0056] In an embodiment of the present application, the location range information of the road-related facilities can be determined in advance based on the first road video data collected by the camera, and then the second road video data collected by the camera is obtained. Based on the second road video data, the location information of the active object is detected, and then the location information of the active object is matched with the location range information of the road-related facilities to determine the matching result, so as to compare the active object of the road with the static road facilities, and analyze whether the interactive object appears at the location of the static road facility where it should not appear. In the case that the matching result meets the offset condition, an offset alarm message is generated and sent to adjust the shooting angle of the camera, so that the offset can be detected and an alarm can be issued in time, and the accuracy of the matching result based on the camera image can be guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a schematic diagram of an example of a road camera and its shooting scene according to an embodiment of the present application;

[0058] Figure 2 This is a flowchart of an embodiment of a method for detecting position offset of a camera according to the present application;

[0059] Figure 3 is a flowchart of another embodiment of a method for detecting position offset of a camera according to the present application;

[0060] Figure 4 is a flowchart of another embodiment of a method for detecting position offset of a camera according to the present application;

[0061] Figure 5 is a flowchart of another embodiment of a method for detecting position offset of a camera according to the present application;

[0062] Figure 6 It is a structural diagram of an exemplary device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0064] The embodiments of the present application can be applied to various scenes captured by cameras, such as road cameras capturing roads, and cameras capturing public areas such as shopping malls and hospitals. Many cameras have adjustable shooting angles to capture videos from different angles. Cameras generally use a rotating angle method, i.e., they switch angles at fixed intervals so that videos from all angles can be captured.

[0065] During the shooting process, if the camera angle shifts, such as a road camera shifting due to wind or other factors, the captured video data will be distorted, causing problems in the recognition results based on the image. Therefore, the embodiment of the present application can determine whether the camera is offset based on the position of static objects and moving objects, and issue an early warning when offset occurs.

[0066] Reference Figure 1 The figure is a schematic diagram of an example of a road camera and its shooting scene. The road has six lanes and a pedestrian crossing, which is captured by a dome-type road camera. The dotted line indicates the shooting range of the camera at the correct angle. Figure 2 As shown:

[0067] Step 202: Acquire location range information of road-related facilities, where the location range information of the road-related facilities is pre-determined based on first road video data captured by a camera.

[0068] The embodiments of the present application determine whether the camera is offset based on the positions of static and moving objects. Therefore, road video data from the camera can be pre-collected, which can be referred to as first road video data. The first road video data can be road video data collected by the camera under favorable imaging conditions. Specifically, the road video data collected by the camera can be screened to determine the road video data whose imaging conditions meet the detection requirements as the first road video data. The detection requirements can be determined based on various conditions such as lighting and occlusion.

[0069] The first road video data can then be analyzed to determine the location range information of the road-related facilities. The first road video data can be analyzed through various methods such as object recognition and semantic segmentation to determine the road-related facilities in the video data and the location range of the road-related facilities.

[0070] Among them, the road-related facilities include: roads and road facilities; the roads include motor vehicle lanes, non-motor vehicle lanes, and sidewalks; the road facilities include at least one of the following: traffic signs, markings, pedestrian overpasses, pedestrian tunnels, separation facilities, road display screens, lighting equipment, and bus stops; separation facilities include at least one of the following: guardrails, bollards, green belts, and flower beds.

[0071] In an optional embodiment, determining the location range information of road-related facilities based on the first road video data captured by a camera in advance includes: obtaining each frame of road imagery from the first road video data; performing semantic segmentation on each frame of road imagery to determine a segmentation result; and determining the location range information of road-related facilities based on the segmentation result. Each frame of road imagery can be obtained sequentially from the first road video data, and then semantic segmentation can be performed on each frame of road imagery to predict the semantic category of each pixel and determine a segmentation result. The segmentation result can include, for example, the segmented target and its location. Based on the segmentation result, the type of road-related facility the segmented target is determined, as well as its location range.

[0072] Among them, semantic segmentation can be achieved through a semantic segmentation model based on deep learning, and the first road video data can be input into the semantic segmentation model for semantic segmentation, and the segmentation result can be output; the location range information of the road-related facilities can be determined based on the segmentation result. The first road video data is input into the semantic segmentation model, and the semantic segmentation model can perform semantic segmentation on each frame of the road image respectively, output the segmentation result, and the location range information of each road-related facility can be determined based on the segmentation result. Among them, image semantic segmentation (Semantic Image Segmentation) is an important part of image understanding (Image understanding) in computer vision (Computer vision). The embodiment of the present application can adopt various deep learning network models to be applied to the semantic segmentation task to achieve the segmentation of road-related facilities. For example, the FCN (Fully Convolution Network) algorithm, SegNet algorithm, DeepLab V3+ algorithm, etc. are adopted. Thereby, each road-related facility can be determined based on semantic segmentation, and the location range information of the road-related facilities can be determined.

[0073] In the embodiment of the present application, the camera can be moved between multiple angles to adjust the shooting range. Therefore, a single angle can be determined as a base angle, and the location range information corresponding to the road-related facilities at that base angle can be used as a reference. Images captured at other angles can then be corrected to the base angle to match the location range information corresponding to the road-related facilities at that base angle. Of course, in other embodiments, each angle can also correspond to the location range information of a road-related facility, and this embodiment of the present application is not limited to this.

[0074] Step 204: Acquire second road video data captured by the camera.

[0075] After the location range information of the road-related facilities has been determined, the angle of the camera can be detected in real time or periodically, so that the second road video data collected by the camera can be obtained in real time, or the second road video data collected by the camera can be detected periodically. The detection period can be less than the angle adjustment period of the camera, so that the angle is detected at least once in each angle adjustment period to see if it is accurate.

[0076] In the embodiment of the present application, the road video data collected by the same camera is divided into first and second types according to the scene. The pre-processed road video data is referred to as the first road video data, and the detected road video data is referred to as the second road video data. The second road video data can be collected at any time and under any imaging conditions. Since the location information of static road-related facilities has been analyzed during pre-processing, even if the second road video data undergoes a significant change in lighting (such as at night or on a cloudy day) or is severely obstructed, it can still be used to detect moving objects, achieving all-day and all-weather detection.

[0077] Step 206: Detect location information of the moving object based on the second road video data.

[0078] Target detection may be performed on the second road video data to determine an active object and position information of the active object. The position information may be a set of position information of position points of the active object.

[0079] In an optional embodiment, detecting the location information of active objects based on the second road video data includes: performing target recognition based on each frame of road image in the second road video data to determine at least one target object; tracking the at least one target object in the second road video data to determine at least one active object; and determining the location information of the at least one active object. Target recognition can be performed based on each frame of road image in the second road video data to determine at least one target object, and then tracking the at least one target object in the second road video data to determine at least one active object; and determining the location information of the at least one active object. In a road scene, active objects on the road include vehicles and pedestrians, wherein vehicles include motor vehicles and non-motor vehicles. Non-motor vehicles and pedestrians are both non-motorized active objects, while motor vehicles are motorized active objects.

[0080] Active object detection can be performed using deep learning methods, such as traffic object detection, and can be implemented using a variety of deep learning algorithms, including region proposal-based object detection and recognition algorithms such as R-CNN, Fast-R-CNN, and Faster-R-CNN, as well as regression-based object detection and recognition algorithms such as YOLO and SSD. The second road video data can be input into a deep learning-based object detection model to identify and track active objects and determine the location information of at least one active object.

[0081] Step 208: Match the location information of the active object with the location range information of the road-related facilities to determine a matching result.

[0082] The location information of the active object can be matched with the location range information of the road-related facilities to determine whether their locations overlap, or whether the active object appears in a location that it should not appear, thereby determining a matching result. Matching the location information of the active object with the location range information of the road-related facilities to determine a matching result includes: comparing the location information of the active object with the location range information of the road-related facilities, counting overlap information where the location information of the active object appears within the location range information of the road-related facilities; and generating a matching result based on the overlap information. The location information of the active object can be compared with the location range information of the road-related facilities to determine whether the location information of the active object overlaps with the location range information of repulsive road-related facilities. Repulsive road-related facilities are facilities where the active object should not move, such as motor vehicle lanes, flower beds, and green belts where pedestrians should not move, or non-motor vehicle lanes, flower beds, and green belts where vehicles should not move. Overlap information, such as the number of overlaps, frequency of overlaps, and duration of overlaps, can be counted as a matching result.

[0083] In an optional embodiment, counting the overlapping information of the location information of the active object appearing within the location range information of the road-related facility includes: counting the number of times the location information of the active object appears within the location range information of the road-related facility within a set time; and counting the overlapping frequency based on the set time and the number of overlapping times. In this embodiment of the application, second video data can be collected at a set time. For example, for video data collected in real time, the matching results can be accumulated up to the set time. Within the set time, the number of overlaps between each active object and the road-related facility can be notified, that is, the number of times the location information of the active object appears within the location range information of the road-related facility, and then the overlapping frequency can be counted based on the set time and the number of overlapping times. Accordingly, each active object can correspond to a corresponding number of overlapping times.

[0084] In this embodiment of the present application, to prevent individual noise from affecting matching accuracy, the overlapping frequencies of various moving objects can also be analyzed and filtered. For example, if the number of moving objects with overlapping frequencies exceeding the matching threshold exceeds a threshold, the overlapping frequencies of each moving object can be included in the matching results. If the threshold is not exceeded, it is considered that some moving objects are noise, such as noise caused by individuals crossing the road, and the noise data can be discarded to obtain the matching results.

[0085] In the embodiment of the present application, the overlap detected is for overlap that should not occur, such as pedestrians walking on flower beds or motor vehicle lanes, or vehicles driving on flower beds and sidewalks. Therefore, the active object is compared with the repulsive road-related facilities. The types of the active objects can include: motorized and non-motorized categories, and the road-related facilities include: motorized and non-motorized categories. Among them, the motorized category refers to road-related facilities associated with the normal operation of motor vehicles, such as motor vehicle lanes and their traffic signs, dividing facilities, road display screens, etc., and the non-motorized category refers to road-related facilities such as non-motorized lanes and their traffic signs, dividing facilities, sidewalks and their traffic signs, pedestrian overpasses, pedestrian tunnels, lighting equipment, bus stops, etc.

[0086] In an optional embodiment, comparing the location information of the active object with the location range information of the road-related facilities includes: comparing the location information of a motorized active object with the location range information of a non-motorized road-related facility to determine whether the two overlap; and / or comparing the location information of a non-motorized active object with the location range information of a motorized road-related facility to determine whether the two overlap. Comparing the location information of a motorized active object with the location range information of a non-motorized road-related facility to determine whether the two overlap can detect whether motorized active objects such as vehicles are traveling in locations such as flower beds or sidewalks. Comparing the location information of a non-motorized active object with the location range information of a motorized road-related facility to determine whether the two overlap can detect whether non-motorized active objects such as pedestrians and non-motorized vehicles are moving in locations such as flower beds or motorway lanes. To eliminate interference from abnormal factors, such as pedestrians jaywalking without using the sidewalk, the number, frequency, and duration of overlap can be determined based on the second road video data over a period of time. Detection can also be performed based on the overlapping information of multiple active objects.

[0087] Step 210: If the matching result satisfies the deviation condition, a deviation alarm message is generated.

[0088] Based on the matching results, it can be determined whether the offset condition is met. If the matching results meet the offset condition, an offset alarm message is generated. Generating the offset alarm message when the matching results meet the offset condition includes: determining whether the overlapping information exceeds an overlap threshold; if the overlapping information exceeds the overlap threshold, determining that the offset condition is met and generating an offset alarm message. Whether the overlapping information exceeds the overlap threshold can be determined, for example, by determining whether the number of overlaps exceeds a number threshold, whether the overlapping frequency exceeds a frequency threshold, or whether the overlapping duration exceeds a duration threshold. If the overlapping information exceeds the overlap threshold, it is determined that the offset condition is met. In other examples, the offset condition can be determined to be met based on the overlapping information of multiple active objects exceeding the overlap threshold, and an offset alarm message can then be generated.

[0089] The camera identifier of the camera that has experienced a deviation can be added to the deviation alarm information so that the camera that has experienced a deviation can be determined based on the deviation alarm information. The deviation information can also be analyzed based on the position information of the overlapping active object and the position range information of the road-related facilities, where the deviation information includes: a deviation direction and / or a deviation angle; and the deviation information can be added to the deviation alarm information. The deviation information can be analyzed based on the position information of the overlapping active object and the position range information of the road-related facilities, for example, the direction of the deviation can be analyzed. This involves comparing the position information of the overlapping active object with the position range information of the road-related facilities, including comparing with relevant and repulsive road-related facilities, thereby determining the deviation position of the active object and its normal (non-deflected) position, thereby detecting the direction and angle of the deviation, etc.

[0090] like Figure 1 In the example shown, the camera's normal range is the dashed line, while the actual range is the dashed line. This indicates that both the vehicle and pedestrian have shifted, with the vehicle moving at the intersection of the motor vehicle and the sidewalk, and the pedestrian moving above the motor vehicle, with a crosswalk located relatively close. Based on the shifted vehicles and pedestrians, the camera can be analyzed to indicate a leftward shift (as shown above). This generates a shift alert.

[0091] Step 212: Send the deviation alarm information to adjust the shooting angle of the camera.

[0092] The deviation alarm information may be sent, for example, to equipment of a camera management department, a server device, etc., so that the shooting angle of the camera can be adjusted in time.

[0093] In summary, the location range information of the road-related facilities can be determined in advance based on the first road video data collected by the camera, and then the second road video data collected by the camera can be obtained. Based on the second road video data, the location information of the active object can be detected, and then the location information of the active object can be matched with the location range information of the road-related facilities to determine the matching result, so as to compare the active object of the road with the static road facilities, and analyze whether the interactive object appears at the location of the static road facility where it should not appear. In the case that the matching result meets the offset condition, an offset alarm message is generated and sent to adjust the shooting angle of the camera, so that the offset can be detected and an alarm can be issued in time, and the accuracy of the matching result based on the camera image can be guaranteed.

[0094] In an embodiment of the present application, the control end of the camera can perform the following processing steps: regularly instruct the camera to adjust the shooting angle; receive offset alarm information; and adjust the shooting angle according to the offset alarm information.

[0095] During camera operation, the camera's control terminal, such as a central control terminal or the camera's own controller, can periodically instruct the camera to adjust its shooting angle. If the camera's shooting angle shifts for various reasons, the detection terminal can detect it using the aforementioned method and then send a shift alarm message. The control terminal receives the shift alarm message and adjusts the shooting angle based on it. For example, if the camera position shifts based on the shift alarm message, the shift angle can be automatically detected and adjusted. Alternatively, the shift angle can be determined and adjusted based on the captured image, etc. Furthermore, the shift alarm message can be used to obtain shift information, such as the direction and angle of the shift, to adjust the camera's shooting angle.

[0096] On the basis of the above-mentioned embodiment, the embodiment of the present application further provides a method for detecting position offset of a camera, which can pre-analyze the position range information of road-related facilities.

[0097] Reference Figure 3 , shows a step flow chart of another camera position offset detection method preprocessing embodiment of the present application.

[0098] Step 302: First road video data with imaging conditions meeting detection requirements is collected through a camera.

[0099] Step 304: Acquire each frame of road image from the first road video data.

[0100] Step 306: Perform semantic segmentation on each frame of the road image to determine a segmentation result.

[0101] The first road video data may be input into a semantic segmentation model for processing, thereby performing semantic segmentation based on each frame of road image and outputting corresponding segmentation results.

[0102] Step 308: Determine the location range information of the road-related facilities based on the segmentation result.

[0103] During the preprocessing process, some static road-related facilities can be detected. Therefore, road video data collected under good imaging conditions can be selected for analysis, and road-related facilities, such as roads and road facilities, can be determined based on deep learning semantic segmentation; the roads include motor vehicle lanes, non-motor vehicle lanes, and sidewalks; the road facilities include at least one of the following: traffic signs, markings, pedestrian overpasses, pedestrian tunnels, separation facilities, road display screens, lighting equipment, and bus stops; separation facilities include at least one of the following: guardrails, bollards, green belts, and flower beds.

[0104] The embodiments of this application can pre-detect static objects such as road-related facilities under favorable imaging conditions, facilitating subsequent matching as a benchmark. Compared to existing methods that store multiple scene images for image matching, this detection method is simpler and more efficient. Furthermore, it does not require custom gimbals or other hardware, resulting in no additional hardware costs.

[0105] On the basis of the above embodiments, the embodiments of the present application further provide a method for detecting position offset of a camera, which can detect the position offset of the camera in a timely manner.

[0106] Reference Figure 4 , shows a step flow chart of another camera position offset detection method preprocessing embodiment of the present application.

[0107] Step 402: Acquire second road video data captured by a camera.

[0108] Step 404 : Perform target recognition based on each frame of road image in the second road video data to determine at least one target object.

[0109] Step 406 : Track the at least one target object in the second road video data to determine at least one moving object.

[0110] Step 408: Determine the location information of the at least one active object.

[0111] Step 410 : Compare the location information of the motorized active object with the location range information of the non-motorized associated road facilities to determine whether the two overlap.

[0112] If there is overlap, step 414 is executed; if there is no overlap, the detection continues and step 410 is executed.

[0113] Step 412 : Compare the location information of the non-motorized active object with the location range information of the motorized road-related facilities to determine whether the two overlap.

[0114] If there is overlap, step 414 is executed; if there is no overlap, the detection continues and step 412 is executed.

[0115] Step 414 : Count the overlapping information of the location information of the active object appearing within the location range information of the road-related facilities.

[0116] Step 416: Generate matching results based on the overlapping information.

[0117] Step 418: Determine whether the overlapping information exceeds the overlapping threshold.

[0118] If yes, execute step 420; if no, end the process.

[0119] If the overlapping information exceeds the overlapping threshold, it is determined that the offset condition is met and offset alarm information is generated.

[0120] Step 420 : Analyze offset information based on the position information of the overlapping active objects and the position range information of the road-related facilities. The offset information includes an offset direction and / or an offset angle.

[0121] Step 422: Add the offset information to the offset alarm information to generate offset alarm information.

[0122] The embodiment of the present application can be segmented based on roads and perform human and vehicle detection, and can realize preset position offset detection of cameras such as dome cameras in traffic scenes all day and all weather. The operation is simple, does not require matching pictures in various scenes, is low-cost, and does not require changes or additional hardware.

[0123] The embodiment of the present application pre-detects the location of static road-related facilities, so that subsequent detection can be performed around the clock and in all weather conditions, and is still applicable when there are significant changes in lighting or severe obstructions.

[0124] The above embodiments take a road camera as an example, and in actual processing, they can also be applied to other scenes captured by cameras.

[0125] Reference Figure 5 , shows a step flow chart of another camera position offset detection method preprocessing embodiment of the present application.

[0126] Step 502: Acquire location range information of a static facility, where the location range information of the static facility is determined based on first video data collected by a camera.

[0127] The method of determining the location range information of the static facility in advance based on the first video data collected by the camera includes: obtaining each frame of video image from the first video data; performing semantic segmentation on each frame of video image to determine the segmentation result; and determining the location range information of the static facility based on the segmentation result.

[0128] Among them, the static facilities can be determined based on the scene captured by the camera. For example, in an indoor scene, the static facilities can be walls, storage equipment and other facilities.

[0129] Step 504: Acquire the second video data captured by the camera.

[0130] Step 506: Detect position information of the moving object based on the second video data.

[0131] The detecting the position information of the moving object based on the second video data includes: performing target recognition based on each frame of video image in the second video data to determine at least one target object; tracking the at least one target object in the second video data to determine at least one moving object; and determining the position information of the at least one moving object.

[0132] Step 508: Match the location information of the active object with the location range information of the static facility to determine a matching result.

[0133] The matching of the location information of the active object with the location range information of the static facility to determine the matching result includes: comparing the location information of the active object with the location range information of the static facility; counting the overlapping information of the location information of the active object within the location range information of the static facility; and generating a matching result based on the overlapping information.

[0134] Step 510: If the matching result satisfies the deviation condition, a deviation alarm message is generated.

[0135] In the case that the matching result satisfies the offset condition, generating offset alarm information includes: determining whether the overlapping information exceeds the overlapping threshold; if the overlapping information exceeds the overlapping threshold, determining that the offset condition is satisfied and generating offset alarm information.

[0136] Based on the position information of the overlapping active objects and the position range information of the static facilities, the offset information is analyzed, the offset information including: offset direction and / or offset angle; and the offset information is added to the offset alarm information.

[0137] Step 512: Send the deviation alarm information to adjust the shooting angle of the camera.

[0138] The embodiments of the present application can segment static facilities based on specific scenarios and perform active object detection, and can realize preset position offset detection of cameras such as dome cameras around the clock and in all weather conditions. The operation is simple, does not require matching pictures in various scenarios, is low-cost, and does not require changes or additions to hardware.

[0139] In each embodiment of the present application, if user information is involved, it is collected, used and stored after obtaining the user's authorization, and various operations based on user information are also performed after obtaining the user's authorization.

[0140] It should be noted that for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the order of the actions described, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0141] On the basis of the above embodiment, this embodiment further provides a camera position offset detection device, which is applied to electronic devices, such as electronic devices at a server end.

[0142] a facility determination module, configured to obtain location range information of road-related facilities, wherein the location range information of the road-related facilities is pre-determined based on first road video data collected by a camera;

[0143] a position detection module, configured to obtain second road video data captured by a camera; and detect position information of a moving object based on the second road video data;

[0144] an offset detection module, configured to match the location information of the moving object with the location range information of the road-related facilities to determine a matching result;

[0145] The alarm module is used to generate an offset alarm message when the matching result meets the offset condition; and send the offset alarm message to adjust the shooting angle of the camera.

[0146] In summary, the location range information of the road-related facilities can be determined in advance based on the first road video data collected by the camera, and then the second road video data collected by the camera can be obtained. Based on the second road video data, the location information of the active object can be detected, and then the location information of the active object can be matched with the location range information of the road-related facilities to determine the matching result, so as to compare the active object of the road with the static road facilities, and analyze whether the interactive object appears at the location of the static road facility where it should not appear. In the case that the matching result meets the offset condition, an offset alarm message is generated and sent to adjust the shooting angle of the camera, so that the offset can be detected and an alarm can be issued in time, and the accuracy of the matching result based on the camera image can be guaranteed.

[0147] The position detection module is used to perform target recognition based on each frame of road image in the second road video data to determine at least one target object; track the at least one target object in the second road video data to determine at least one active object; and determine the position information of the at least one active object.

[0148] The offset detection module is configured to compare the location information of the moving object with the location range information of the road-related facilities; count the overlapping information of the location information of the moving object within the location range information of the road-related facilities; and generate a matching result based on the overlapping information.

[0149] Among them, the types of active objects include: motorized and non-motorized types, and the road-related facilities include: motorized and non-motorized types; the offset detection module is used to compare the position information of motorized active objects with the position range information of non-motorized road-related facilities to determine whether the two overlap; and / or, compare the position information of non-motorized active objects with the position range information of motorized road-related facilities to determine whether the two overlap.

[0150] The alarm module is used to determine whether the overlapping information exceeds the overlapping threshold; if the overlapping information exceeds the overlapping threshold, it is determined that the offset condition is met and an offset alarm message is generated.

[0151] The alarm module is further configured to analyze offset information based on the position information of the overlapping active objects and the position range information of the road-related facilities, wherein the offset information includes: an offset direction and / or an offset angle;

[0152] The offset information is added to the offset alarm information.

[0153] The facility determination module is used to obtain each frame of road image from the first road video data; perform semantic segmentation on each frame of road image to determine a segmentation result; and determine the location range information of road-related facilities based on the segmentation result.

[0154] The road-related facilities include: roads and road facilities; the roads include motor vehicle lanes, non-motor vehicle lanes, and sidewalks; the road facilities include at least one of the following: traffic signs, markings, pedestrian overpasses, pedestrian tunnels, separation facilities, road display screens, lighting equipment, and bus stops; separation facilities include at least one of the following: guardrails, bollards, green belts, and flower beds.

[0155] The embodiment of the present application can be segmented based on roads and perform human and vehicle detection, and can realize preset position offset detection of cameras such as dome cameras in traffic scenes all day and all weather. The operation is simple, does not require matching pictures in various scenes, is low-cost, and does not require changes or additional hardware.

[0156] The embodiment of the present application pre-detects the location of static road-related facilities, so that subsequent detection can be performed around the clock and in all weather conditions, and is still applicable when there are significant changes in lighting or severe obstructions.

[0157] On the basis of the above embodiment, this embodiment further provides another camera position offset detection device, which is applied to electronic devices, such as electronic devices at a server end.

[0158] a preprocessing module, configured to obtain location range information of a static facility, wherein the location range information of the static facility is determined based on first video data collected by a camera;

[0159] An activity detection module is configured to obtain second video data captured by a camera; and detect location information of an active object based on the second video data;

[0160] an offset analysis module, configured to match the location information of the active object with the location range information of the static facility to determine a matching result;

[0161] The offset alarm module is used to generate offset alarm information when the matching result meets the offset condition; and send the offset alarm information to adjust the shooting angle of the camera.

[0162] In each embodiment of the present application, if user information is involved, it is collected, used and stored after obtaining the user's authorization, and various operations based on user information are also performed after obtaining the user's authorization.

[0163] An embodiment of the present application further provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute instructions (instructions) of each method step in the embodiment of the present application.

[0164] The present application provides one or more machine-readable media having instructions stored thereon, which, when executed by one or more processors, cause an electronic device to perform one or more of the methods described in the above embodiments. In the present application, the electronic device includes a server, a terminal device, and the like.

[0165] The embodiments of the present disclosure may be implemented as a device configured as desired using any appropriate hardware, firmware, software, or any combination thereof, and the device may include electronic devices such as a server (cluster), a terminal, etc. Figure 6 An exemplary apparatus 600 that may be used to implement various embodiments described herein is schematically illustrated.

[0166] For one embodiment, Figure 6 An exemplary apparatus 600 is shown having one or more processors 602, a control module (chip set) 604 coupled to at least one of the processor(s) 602, a memory 606 coupled to the control module 604, a non-volatile memory (NVM) / storage device 608 coupled to the control module 604, one or more input / output devices 610 coupled to the control module 604, and a network interface 612 coupled to the control module 604.

[0167] The processor 602 may include one or more single-core or multi-core processors, and the processor 602 may include any combination of general-purpose processors or dedicated processors (e.g., graphics processors, application processors, baseband processors, etc.). In some embodiments, the apparatus 600 can serve as a server, terminal, or other device described in the embodiments of the present application.

[0168] In some embodiments, the apparatus 600 may include one or more computer-readable media (e.g., memory 606 or NVM / storage 608) having instructions 614 and one or more processors 602 configured in conjunction with the one or more computer-readable media to execute the instructions 614 to implement a module to perform the actions described in the present disclosure.

[0169] For one embodiment, the control module 604 may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) 602 and / or any suitable device or component in communication with the control module 604 .

[0170] The control module 604 may include a memory controller module to provide an interface to the memory 606. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0171] The memory 606 can be used, for example, to load and store data and / or instructions 614 for the device 600. For one embodiment, the memory 606 can include any suitable volatile memory, such as a suitable DRAM. In some embodiments, the memory 606 can include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0172] For one embodiment, the control module 604 may include one or more input / output controllers to provide interfaces to the NVM / storage device 608 and the input / output device(s) 610 .

[0173] For example, NVM / storage 608 may be used to store data and / or instructions 614. NVM / storage 608 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0174] NVM / storage device 608 may include storage resources that are part of the device on which apparatus 600 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 608 may be accessible via input / output device(s) 610 over a network.

[0175] (One or more) input / output devices 610 may provide an interface for the apparatus 600 to communicate with any other appropriate device. The input / output devices 610 may include communication components, audio components, sensor components, etc. The network interface 612 may provide an interface for the apparatus 600 to communicate via one or more networks. The apparatus 600 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, accessing a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof for wireless communication.

[0176] For one embodiment, at least one of the processor(s) 602 may be packaged together with the logic of one or more controllers of the control module 604 (e.g., a memory controller module). For one embodiment, at least one of the processor(s) 602 may be packaged together with the logic of one or more controllers of the control module 604 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 602 may be integrated on the same die with the logic of one or more controllers of the control module 604. For one embodiment, at least one of the processor(s) 602 may be integrated on the same die with the logic of one or more controllers of the control module 604 to form a system-on-chip (SoC).

[0177] In various embodiments, the apparatus 600 may be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, the apparatus 600 may have more or fewer components and / or a different architecture. For example, in some embodiments, the apparatus 600 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touchscreen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0178] Among them, the main control chip can be used as a processor or control module in the detection device, sensor data, location information, etc. are stored in the memory or NVM / storage device, the sensor group can be used as an input / output device, and the communication interface may include a network interface.

[0179] The present application also provides an electronic device comprising: a processor; and a memory storing executable code, wherein when the executable code is executed, the processor executes one or more methods described in the embodiments of the present application. The memory in the embodiments of the present application can store various data, such as target files, file-application association data, and other data, and can also include user behavior data, thereby providing a data foundation for various processing.

[0180] The embodiments of the present application further provide one or more machine-readable media on which executable codes are stored. When the executable codes are executed, the processor executes one or more methods described in the embodiments of the present application.

[0181] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0182] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0183] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0184] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0185] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0186] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0187] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0188] The above is a detailed introduction to a camera position offset detection method and device, an electronic device and a storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for detecting position offset of a camera, characterized in that: The method comprises: Acquiring location range information of road-related facilities, where the location range information of the road-related facilities is pre-determined based on first road video data collected by a camera; the road-related facilities include: roads and road facilities; the road facilities include: lighting equipment; Obtaining second road video data collected by the camera; detecting position information of a moving object based on the second road video data; comparing the location information of the active object with the location range information of the road-related facility; Counting overlapping information of the location information of the active object appearing within the location range information of the road-related facility, the overlapping information at least including: overlapping frequency, wherein if the number of active objects whose overlapping frequency exceeds a matching threshold does not exceed a number threshold, the active object is discarded as noise data; Generate matching results based on overlapping information; When the matching result satisfies the offset condition, generating offset alarm information; The deviation alarm information is sent to adjust the shooting angle of the camera.

2. The method according to claim 1, characterized in that The detecting the position information of the moving object based on the second road video data includes: performing target recognition based on each frame of road image in the second road video data to determine at least one target object; Tracking the at least one target object in the second road video data to determine at least one moving object; Position information of the at least one active object is determined.

3. The method according to claim 1, characterized in that The types of the active objects include: motorized and non-motorized types; the road-related facilities include: motorized and non-motorized types; The comparing the location information of the active object with the location range information of the road-related facility includes: Comparing the location information of the motorized activity object with the location range information of the non-motorized associated road facilities to determine whether the two overlap; and / or, Compare the location information of non-motorized activity objects with the location range information of motorized road-related facilities to determine whether the two overlap.

4. The method according to claim 2 or 3, characterized in that The counting of overlapping information of the location information of the active object appearing within the location range information of the road-related facility includes: Counting the number of times that the location information of the active object appears within the location range information of the road-related facility within a set time; The overlapping frequency is counted according to the set time and the number of overlapping times.

5. The method according to claim 4, characterized in that When the matching result satisfies the deviation condition, a deviation alarm message is generated, including: Determine whether the overlapping information exceeds the overlapping threshold; If the overlapping information exceeds the overlapping threshold, it is determined that the offset condition is met and an offset alarm message is generated.

6. The method according to claim 5, characterized in that Also includes: Analyzing offset information based on the position information of the overlapping active objects and the position range information of the road-related facilities, the offset information including: an offset direction and / or an offset angle; The offset information is added to the offset alarm information.

7. The method according to claim 1, characterized in that Also includes: The step of determining the location range information of the road-related facilities in advance based on the first road video data collected by the camera: Acquire each frame of road image from the first road video data; Performing semantic segmentation on each frame of road image to determine a segmentation result; The location range information of the road-related facilities is determined according to the segmentation result.

8. The method according to claim 1, characterized in that Also includes: The step of determining the location range information of the road-related facilities based on the first road video data collected by the camera in advance: Inputting the first road video data into the semantic segmentation model for semantic segmentation, and outputting the segmentation result; The location range information of the road-related facilities is determined according to the segmentation result.

9. The method according to any one of claims 1 to 3, characterized in that: The road includes motor vehicle lanes, non-motor vehicle lanes, and sidewalks; the road facilities also include at least one of the following: traffic signs, markings, pedestrian overpasses, pedestrian tunnels, separation facilities, road display screens, and bus stops; the separation facilities include at least one of the following: guardrails, bollards, green belts, and flower beds.

10. A method for detecting position offset of a camera, characterized in that: The method comprises: Acquiring location range information of a static facility, where the location range information of the static facility is determined based on first video data collected by a camera; Acquire second video data captured by the camera; detecting position information of a moving object based on the second video data; comparing the location information of the active object with the location range information of the static facility; Counting overlapping information of the location information of the active object appearing within the location range information of the static facility, the overlapping information at least including: overlapping frequency, wherein if the number of active objects whose overlapping frequency exceeds a matching threshold does not exceed a number threshold, the active object is discarded as noise data; Generate matching results based on overlapping information; When the matching result satisfies the offset condition, generating offset alarm information; The deviation alarm information is sent to adjust the shooting angle of the camera.

11. A camera position deviation detection device, characterized in that: The device comprises: a facility determination module, configured to obtain location range information of road-related facilities, the location range information of the road-related facilities being pre-determined based on first road video data collected by a camera; the road-related facilities including: roads and road facilities; the road facilities including: lighting equipment; a position detection module, configured to obtain second road video data captured by a camera; and detect position information of a moving object based on the second road video data; an offset detection module, configured to compare the location information of the active object with the location range information of the road-related facility; to count overlapping information where the location information of the active object appears within the location range information of the road-related facility, the overlapping information including at least an overlapping frequency; wherein if the number of active objects whose overlapping frequency exceeds a matching threshold does not exceed a number threshold, the active object is discarded as noise data; and to generate a matching result based on the overlapping information; The alarm module is used to generate an offset alarm message when the matching result meets the offset condition; and send the offset alarm message to adjust the shooting angle of the camera.

12. An electronic device comprising: processor; and a memory having executable codes stored thereon, wherein when the executable codes are executed by a processor, the method according to any one of claims 1 to 10 is executed.

13. One or more machine-readable media having executable codes stored thereon, which, when executed by a processor, perform the method according to any one of claims 1 to 10.

Images