Installation abnormality detection method, installation abnormality detection program, and installation abnormality detection device

Abstract

To provide a technology capable of detecting positional deviation of a camera without requiring any devices such as a gyro sensor.SOLUTION: An exemplary installation abnormality detection method for detecting positional deviation of a camera comprises: calculating a positional relation index value indicating a positional relation between a first specific location and a second specific location in an object image included in an image captured by the camera; comparing the calculated positional relation index value with a standard index value for the first specific location and the second specific location being set for an object in the object image; and detecting the positional deviation of the camera based on the comparison result between the positional relation index value and the standard index value.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a technique for detecting a positional deviation of a camera. [Background technology]

[0002] Object detection methods using Deep Neural Networks (DNNs) generally have a problem in that detection accuracy decreases when detection data is acquired in an environment (domain) different from that of the training data. This decrease in detection accuracy is caused by differences in data distribution due to differences in the environment.

[0003] For example, in an object detection model that uses images captured by a fixed camera, there is a problem in that detection accuracy decreases when the installation conditions of the camera differ between the training data and the detection data. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-46928 A Summary of the Invention [Problem to be solved by the invention]

[0005] The technology proposed in Patent Document 1 makes it possible to update an object detection model depending on the installation conditions of the camera.

[0006] However, providing the camera with a device such as a gyro sensor to detect changes in the installation conditions of the camera, i.e., to detect a positional deviation of the camera, increases the cost of the camera.

[0007] In view of the above, an object of the present invention is to provide a technique capable of detecting a positional deviation of a camera without requiring a device such as a gyro sensor. [Means for solving the problem]

[0008] An exemplary installation anomaly detection method of the present invention is a method for detecting a positional misalignment of a camera, which calculates a positional relationship index value indicating the positional relationship between a first specific location and a second specific location in an object image included in an image captured by the camera, compares the calculated positional relationship index value with standard index values ​​for the first specific location and the second specific location set for an object in the object image, and detects a positional misalignment of the camera based on the comparison result between the positional relationship index value and the standard index value. Effect of the Invention

[0009] According to an exemplary embodiment of the present invention, it is possible to detect a positional deviation of a camera without requiring a device such as a gyro sensor. [Brief description of the drawings]

[0010] [Figure 1A] Diagram showing the outline of the object detection system [Figure 1B] Block diagram showing the configuration of a camera displacement detection device [Diagram 2] FIG. 1 is a diagram showing the relationship between a shooting state and a shot image. [Diagram 3] A schematic diagram showing an image captured by a camera. [Figure 4] FIG. 2 is a diagram illustrating an example of a coefficient table; [Diagram 5] A flowchart showing an example of a camera displacement detection method. [Figure 6] FIG. 1 is a diagram showing an image including an object detection result; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings.

[0012] <1. System Overview> 1A is a diagram showing an overview of an object detection system 100 according to an embodiment of the present invention. The object detection system 100 includes a camera 1, an object detection device 2, and a camera deviation detection device 3.

[0013] The camera 1 is fixed to a structure 4 such as a traffic light or a pedestrian bridge. The camera 1 captures an overhead image of the ground 5 at an angle of view θ1. The camera 1 can capture images continuously over time. In other words, the camera 1 can capture so-called video. The images captured by the camera 1 are transmitted from the camera 1 to the object detection device 2.

[0014] The object detection device 2 is configured by a computer device. The object detection device 2 may be a physical computer device or a cloud computer device.

[0015] The object detection device 2 detects objects appearing in an image captured by the camera 1. The object detection device 2 can detect people appearing in an image captured by the camera 1. More specifically, the object detection device 2 can detect the whole body and head of a person appearing in an image captured by the camera 1.

[0016] The object detection device 2 may detect an object using an object detection model constructed by learning artificial intelligence, or may detect an object by, for example, pattern matching without using an object detection model constructed by learning artificial intelligence.

[0017] The camera deviation detection device 3 is configured by a computer device. The camera deviation detection device 3 may be a physical computer device or a cloud computer device. The object detection device 2 and the camera deviation detection device 3 may be configured by the same computer device or may be configured by separate computer devices.

[0018] The camera displacement detection device 3 is an installation abnormality detection device that detects a positional displacement of the camera 1 by utilizing the detection result of the object detection device 2. The positional displacement of the camera 1 means that the positional relationship between the camera 1 and the ground 5 is displaced from the initial installation of the camera 1 due to a change in the fixed state of the camera 1 relative to the structure 4 (displacement of the fixed position of the camera bracket, etc. due to the application of force such as a collision with an object or a strong wind) or deformation of the structure 4 (displacement of the fixed position of the camera support, etc. due to the collision with an object or the application of force such as a strong wind) or the like.

[0019] <2. Details of the camera displacement detection device> [2-1. Configuration of camera displacement detection device] Fig. 1B is a block diagram showing the configuration of camera shift detection device 3. As shown in Fig. 1B, camera shift detection device 3 includes a controller 31. Camera shift detection device 3 further includes a memory unit 32 and a communication unit 33. Note that camera shift detection device 3 may include at least one of an input device such as a keyboard and an output device such as a display, or both.

[0020] The controller 31 includes an arithmetic circuit that performs arithmetic processing. More specifically, the controller 31 includes a processor that performs arithmetic processing and the like. The processor includes, for example, a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The controller 31 may be configured with one processor or multiple processors. When the controller 31 is configured with multiple processors, the processors are connected to each other so as to be able to communicate with each other.

[0021] The memory unit 32 includes a volatile memory and a non-volatile memory. The volatile memory may include, for example, a random access memory (RAM). The non-volatile memory may include, for example, a read only memory (ROM), a flash memory, or a hard disk drive. The non-volatile memory stores computer-readable programs and data. In this embodiment, the memory unit 32 stores a camera deviation detection program 321, a coefficient table 322, and a standard index value 323.

[0022] The communication unit 33 is configured as a communication interface having an interface circuit for communicating with the object detection device 2 and the like.

[0023] The functions of the controller 31 include an image acquisition unit 311, a calculation unit 312, and a comparison unit 313. In this embodiment, the functions of the controller 31 are realized by a processor executing arithmetic processing in accordance with a camera deviation detection program 321 stored in the memory unit 32.

[0024] The camera displacement detection program 321 is an installation abnormality detection program that detects a positional displacement of the camera. The camera displacement detection program 321 may be composed of a single program or multiple programs. That is, the image acquisition unit 311, the calculation unit 312, and the comparison unit 313 may be realized by one program, or may be realized by multiple programs, such as by having each functional unit be realized by a separate program. The image acquisition unit 311, the calculation unit 312, and the comparison unit 313 may be realized as separate devices. As described above, the image acquisition unit 311, the calculation unit 312, and the comparison unit 313 may each be realized by having a processor execute a program, that is, by software, or may be realized by other methods. The image acquisition unit 311, the calculation unit 312, and the comparison unit 313 may each be realized by using, for example, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like. That is, each of the image acquisition unit 311, the calculation unit 312, and the comparison unit 313 may be realized by hardware using a dedicated IC or the like. Also, each of the image acquisition unit 311, the calculation unit 312, and the comparison unit 313 may be realized by a combination of software and hardware. Also, the image acquisition unit 311, the calculation unit 312, and the comparison unit 313 are conceptual components. A function executed by one component may be distributed to multiple components. Also, functions possessed by multiple components may be integrated into one component.

[0025] [2-2. Initial settings for camera displacement detection device] When the camera 1 is installed, the coefficient table 322 and the standard index value 323 are set to contents corresponding to the installation status (initial installation status) of the camera 1 by data input via the communication unit 33, the input device, etc. In other words, when the camera 1 is installed, the coefficient table 322 and the standard index value 323 whose contents correspond to the installation status (initial installation status) of the camera 1 are written to the memory unit 32 by data input via the communication unit 33, the input device, etc. The standard index value 323 is a value specific to the object type of the object, and in this embodiment, is a value specific to a person.

[0026] Fig. 2 is a diagram showing the relationship between a shooting state and a captured image. Fig. 2 shows a shooting state and a captured image 6 when a person is photographed with the center of the person's body BC1 coinciding with the center of the angle of view of the camera 1.

[0027] When the installation state of the camera 1 is normal, the distance between the center BC1 of the person's whole body and the camera 1 is D1, and the distance between the center BC1 of the person's whole body and the center HC1 of the person's head in the captured image 6 is d1. When the installation state of the camera 1 is abnormal (camera deviation occurs), the distance between the center BC1 of the person's whole body and the camera 1 is D2, which is different from the distance D1. As shown in the example of FIG. 2, when the distance D2 is greater than the distance D1, the subject appears smaller due to the camera deviation, so that the distance between the center BC1 of the person's whole body and the center HC1 of the person's head in the captured image 6 is d2, which is smaller than the distance d1. In other words, the installation state of the camera 1 affects the distance between the first specific location (e.g., the center BC1 of the whole body) and the second specific location (e.g., the center HC1 of the head) of the object (e.g., the person) in the captured image 6. Using this relationship, the camera deviation detection device 3 detects the position deviation of the camera 1 from the distance between the first specific location (e.g., the center BC1 of the whole body) and the second specific location (e.g., the center HC1 of the head) of the object (e.g., the person) in the captured image 6.

[0028] FIG. 3 is a diagram showing a schematic diagram of an image captured by the camera 1. The image shown in FIG. 3 is divided into three regions R1 to R3 according to the installation condition (initial installation condition) of the camera 1. The correspondence between the regions R1 to R3 and the coordinates of the image is stored in the memory unit 32. The division shown in FIG. 3 is an example, and the number of regions may be less than three or may be four or more. In addition, the regions may not only be divided vertically, but also horizontally or vertically and horizontally according to the installation location and angle of the camera and the characteristics of the camera. The guide distance (approximate distance) L from the camera 1 to the object in each region is also appropriately set according to the installation conditions of the camera 1 and the like.

[0029] Region R1 is a region where the target distance (approximate distance) L from camera 1 to the object is 200 [cm] when the object is located within region R1 on the image. Region R2 is a region where the target distance L from camera 1 to the object is 400 [cm] when the object is located within region R2 on the image. Region R3 is a region where the target distance L from camera 1 to the object is 500 [cm] when the object is located within region R3 on the image.

[0030] In determining which region the object is located in on the image, the definition of the object's position on the image may be, for example, the center of the object (the center of the whole body if the object is a person), for example, the upper end of the object in the vertical direction (the top or center of the head if the object is a person), or for example, the lower end of the object in the vertical direction (the toes if the object is a person). In determining which region the object is located on the image, if the definition of the object's position on the image is, for example, the center of the object, a specific process is to use a bounding box that is widely used in object detection methods to define the center of a rectangular diagram (bounding boxes B1, B2) that surrounds the object (the whole body of a person) detected by the object detection device 2 as the center of the object. For object T1 shown in the image shown in FIG. 3, the position of the object center CN1 on the image is within region R3, and for object T2 shown in the image shown in FIG. 3, the position of the object center CN2 on the image is within region R2.

[0031] 4 is a diagram illustrating an example of the coefficient table 332. In the coefficient table 332, area data Te, a reference distance Wo, a correction coefficient A, and a weighting coefficient B are stored.

[0032] The area data Te is data indicating an area on the image. The area data Te is a primary key in the coefficient table 332, and a data record is formed for each piece of area data Te (for each area), and the estimated distance Wo, correction coefficient A, and weighting coefficient B corresponding to the data record are stored. The estimated distance Wo is data of an estimated distance for an object existing in the corresponding area. The correction coefficient A is data of a correction coefficient used in a calculation process for an object existing in the corresponding area. The weighting coefficient B is data of a weighting coefficient used in a calculation process for an object existing in the corresponding area. That is, in the data example shown in FIG. 4, for an object located in the area R1, 200 cm is applied as the estimated distance Wo from the camera to the object, 0.75 is applied as the correction coefficient A, and 0.8 is applied as the weighting coefficient B.

[0033] The positional relationship on an image between a first specific location and a second specific location included in the same object changes depending on the distance between the object and the camera, even if the object is the same. In other words, when the reference distance L from camera 1 to the object is short, the object appears large in the image captured by camera 1. On the other hand, when the reference distance L from camera 1 to the object is long, the object appears small in the image captured by camera 1, so that the positional relationship on an image between the first specific location and the second specific location included in the same object becomes closer to each other as the distance from camera 1 to the object increases. The correction coefficient A is a coefficient for equalizing this characteristic. In this example, the correction coefficient A is set for each area on the image. When correction is performed on the positional relationship on an image between the first specific location and the second specific location included in the same object using the correction coefficient A, the value of the correction coefficient A is set so that the relative distance on the image between the first specific location and the second specific location according to the position of the object is equalized.

[0034] The weighting coefficient B is a coefficient for determining which region of an object on an image is to be emphasized when the camera deviation detection device 3 detects the positional deviation of the camera 1. In the example shown in FIG. 4, when the camera deviation detection device 3 detects the positional deviation of the camera 1, the object whose position on the image is within the region R3 is most important (weighting coefficient B=1.3), and the object whose position on the image is within the region R1 is least important (weighting coefficient B=0.8). For example, contrary to the example shown in FIG. 4, when the camera deviation detection device 3 detects the positional deviation of the camera 1, the object whose position on the image is within the region R3 is most important, and the object whose position on the image is within the region R1 is most important. For example, irregular situations may occur depending on the installation position and angle of the camera 1, such as the camera being determined to be in a distant region (R3) even though the distance between the camera and the object is close. If the frequency of occurrence of such irregular situations varies depending on the region, the region to be emphasized can be changed according to the frequency of occurrence. That is, each data in coefficient table 332 is set by the designer and developer of object detection system 100, the installer of camera 1, etc., according to the installation conditions of the camera, etc.

[0035] [2-3. Normal operation of the camera displacement detection device] Fig. 5 is a flowchart showing an example of a camera shift detection method executed by the camera shift detection device 3. The camera shift detection method is a method for detecting an installation abnormality that detects a position shift of the camera. While the object detection system 100 is in operation, the camera shift detection device 3 repeatedly executes the camera shift detection method shown in Fig. 5, and the repetition frequency of the process is preferably at an interval appropriate for checking camera shift (for example, once a day, although this varies depending on the use of the camera-captured images), and the timing is preferably a time period when the shooting environment is good (for example, a time period during the day when the density of people passing by is appropriate).

[0036] 5 for each frame of image sent from the object detection device 2, or may execute the camera shift detection method shown in Fig. 5 every few frames for the images sent from the object detection device 2. Note that this image sampling interval is preferably an interval at which the same object is not photographed twice (passes through the shooting range), for example, 10 seconds (which differs depending on the shooting range and the moving speed of the object).

[0037] In step S1, the image acquisition unit 311 (controller 31) determines whether or not the image sampling timing has arrived. When the image sampling timing has arrived, the process proceeds to the next step S2.

[0038] In step S2, the image acquisition unit 311 (controller 31) acquires an image (data) including the object detection result transmitted from the object detection device 2 to the camera shift detection device 3. The object detection result is included in, for example, the metadata of the image. Note that if the object detection result is no detection, the positional relationship index value cannot be calculated in step S6, and therefore the camera shift detection device 3 can skip the processes of steps S3 to S9.

[0039] Fig. 6 is a diagram showing a schematic image including an object detection result. The object detection device 2 can detect a person, and more specifically, can detect the whole body and head of the person. Therefore, in an image showing a full-body image of a person as shown in Fig. 6, a first bounding box BBX1 indicating the position of the person's whole body and a second bounding box BBX2 indicating the position of the person's head are formed.

[0040] The object detection method (bounding box generation method) can use various known image recognition methods (bounding box generation methods). For example, the object detection device 2 can be realized by an AI (artificial intelligence) model trained with learning data based on image data of a person (data consisting of input data (person image data) and correct answer data (data on whether the image is a person or not, data showing the overall position of the person in the image, and data showing the head position of the person in the image)).

[0041] For example, the object detection result includes the center coordinate C1 of the first bounding box BBX1, the horizontal length of the first bounding box BBX1, the vertical length of the first bounding box BBX1, the center coordinate C2 of the second bounding box BBX2, the horizontal length of the second bounding box BBX2, and the vertical length of the second bounding box BBX2. Alternatively, the object detection result may not include the center coordinate C1 of the first bounding box BBX1 and the center coordinate C2 of the second bounding box BBX2, and the camera deviation detection device 3 may calculate the center coordinate C1 of the first bounding box BBX1 and the center coordinate C2 of the second bounding box BBX2 based on the object detection result (analysis of an image in which a bounding box is displayed, or the coordinates of each of the four corners of the bounding box).

[0042] When acquisition of image data including the object detection result is completed, the process proceeds to the next step S3.

[0043] In step S3, the calculation unit 312 (controller 31) generates a first bounding box BBX1 indicating the position of the person's entire body based on the object detection result included in the image (data). In step S4 following step S3, the calculation unit 312 (controller 31) generates a second bounding box BBX2 indicating the position of the person's head based on the object detection result included in the image (data).

[0044] In step S5 following step S4, the calculation unit 312 (controller 31) determines the positions of the first bounding box BBX1 and the second bounding box BBX2. For example, the calculation unit 312 (controller 31) recognizes the center coordinates C1 of the first bounding box BBX1 and the center coordinates C2 of the second bounding box BBX2.

[0045] In step S6 following step S5, the calculation unit 312 (controller 31) calculates the Euclidean distance d between the center coordinates C1 of the first bounding box BBX1 and the center coordinates C2 of the second bounding box BBX2.

[0046] In step S7 following step S6, the calculation unit 312 (controller 31) determines in which area the position of the object on the image is located. In detail, the calculation unit 312 (controller 31) determines in which area (R1, R2, R3) the center coordinate C1 of the first bounding box BBX1 is located according to the installation state of the camera 1. In addition, when multiple objects are detected, the above determination is performed for each object. In step S8 following step S7, the calculation unit 312 (controller 31) searches for the correction coefficient A and weighting coefficient B corresponding to each object using the determination result in step S7 and the coefficient table 332. In step S9 following step S8, the calculation unit 312 (controller 31) associates the Euclidean distance d calculated in step S6 with the correction coefficient A and weighting coefficient B searched in step S7 for each object and stores them in the memory unit 32.

[0047] In step S10 following step S9, the calculation unit 312 (controller 31) determines whether the image data collection period has been completed, i.e., whether the image data collection period has elapsed since the start of the flow operation shown in Fig. 5. If the image data collection period has not been completed, the process returns to step S1. On the other hand, if the image data collection period has been completed, the process proceeds to step S11. The image data collection period may be set so that an image is acquired once in step S2, or may be set so that an image is acquired multiple times in step S2.

[0048] A positional relationship index value indicating a positional relationship on an image between a first specific location and a second specific location included in the same object detected in an image captured by the camera 1 is calculated. More specifically, the calculation unit 312 (controller 31) calculates a positional relationship index value W which is a multiplication value of the Euclidean distance d between the center coordinate C1 of the first bounding box BBX1 and the center coordinate C2 of the second bounding box BBX2 and a correction coefficient A according to the position of the object (person) on the image. Then, the calculation unit 312 (controller 31) calculates an averaged index value AW by performing an averaging process on the calculated multiple positional relationship index values ​​W using a weighting coefficient B according to the position of the object (person) on the image, and regards the calculated averaged index value AW as the positional relationship index value W. The following formula is used to calculate the averaged index value AW. Note that N is the number of objects (people) detected in one or multiple images acquired in step S2 by executing the processes of steps S1 to S10.

number

[0049] When positional relationship index value W is calculated using correction coefficient A as described above, it becomes unnecessary to standardize the reference distance from camera 1 to an object to calculate positional relationship index value W. Therefore, the number of objects (people) that can be used to calculate positional relationship index value W increases, and the probability that positional relationship index value W cannot be calculated (the probability that the above N becomes 0) can be reduced.

[0050] Furthermore, when the positional relationship index value W is calculated by averaging (weighted averaging) using the weighting coefficient B as described above, it is possible to determine in which region on the image an object is to be emphasized when the camera misalignment detection device 3 detects a positional misalignment of the camera 1. Therefore, for example, if there is a direction in which a positional misalignment of the camera 1 is likely to occur, and the position of the camera 1 is shifted in the direction in which the positional misalignment is likely to occur, the accuracy of detecting a positional misalignment of the camera 1 can be improved by emphasizing an object in a region in which a significant change in the positional relationship index value W due to the shift occurs.

[0051] When the calculation of the averaged index value AW is completed, the process proceeds to the next step S12.

[0052] In step S12, the comparison unit 313 reads (extracts) the standard index value 323 corresponding to the object (person) from the memory unit 32. In step S13 following step S12, the comparison unit 313 compares the positional relationship index value W with the standard index value 323. In step S14 following step S13, the comparison unit 313 stores the comparison result in the memory unit 32. Then, in step S15, the comparison unit 313 determines whether the positional relationship index value W and the standard index value 323 do not match for a certain period of time. This certain period corresponds to, for example, a case in which a state in which the positional relationship index value W and the standard index value 323 do not match continues a certain number of times in the flow shown in FIG. 5 which is executed multiple times sequentially over time. Specifically, it is determined whether the positional relationship index value W and the standard index value 323 do not match, and if it is determined that they do not match, the count of the number of times of mismatch is incremented by 1, and if it is determined that they match, the count of the number of times of mismatch is reset (to 0). Thereafter, the determination of mismatch for a certain period of time is made by determining whether the count number of mismatches exceeds a certain number, and determining whether a mismatch has occurred for a certain period of time. Furthermore, when the difference between positional relationship index value W and standard index value 323 is within the allowable error, comparison unit 313 determines that positional relationship index value W and standard index value 323 match, and performs the above determination process.

[0053] If the comparison unit 313 determines that the positional relationship index value W and the standard index value 323 have not matched for a certain period of time, it notifies those around the camera shift detection device 3 that a positional shift of the camera 1 has been detected, using the communication unit 33, an output device, etc., and then ends the flow in Fig. 5. This makes it possible to take measures against the positional shift of the camera 1 outside the camera shift detection device 3. On the other hand, if the comparison unit 313 does not determine that the positional relationship index value W and the standard index value 323 have not matched for a certain period of time, it ends the flow in Fig. 5 without making the above notification.

[0054] The camera shift detection device 3 utilizes the fact that the positional relationship index value W changes when a positional shift of the camera 1 occurs, and detects a positional shift of the camera 1 by capturing the change in the positional relationship index value W. In other words, the camera shift detection device 3 can detect a positional shift of the camera 1 without requiring a device such as a gyro sensor. Also, as long as an object related to the positional relationship index value W is detected, the camera shift detection device 3 can detect a positional shift of the camera 1 in real time.

[0055] In this embodiment, the whole body of a person is adopted as the first specific part of an object detected by the object detection device 2, and the head of the person is adopted as the second specific part of an object detected by the object detection device 2. The movable range of the head of a person is narrower than the movable range of the limbs of a person, and the positional relationship between the whole body of a person and the head of the person changes little due to the movement of the person. Therefore, when the whole body of a person and the head of the person are adopted, it becomes easier to set the standard index value 323. For example, the positional relationship between the whole body of a person and the base of the arm also changes little due to the movement of the person. However, the base of the arm is less likely to show features on an image than the head of the person, so that object detection is more difficult than the head of the person. In other words, the ease of object detection and the ease of setting the standard index value 323 can be satisfied by adopting the whole body of a person as the first specific part of an object detected by the object detection device 2 and adopting the head of the person as the second specific part of an object detected by the object detection device 2.

[0056] Furthermore, in this embodiment, the positional relationship index value W is calculated using the Euclidean distance d between the center coordinates C1 of the first bounding box BBX1 and the center coordinates C2 of the second bounding box BBX2. Since a format including the center coordinates of the bounding boxes is relatively versatile as a format for the object detection result, in this embodiment, it is possible to simplify the calculation of the positional relationship index value W.

[0057] In addition, instead of the Euclidean distance d between the center coordinate C1 of the first bounding box BBX1 and the center coordinate C2 of the second bounding box BBX2, the relationship between the size of the first bounding box BBX1 and the size of the second bounding box BBX2 may be used.

[0058] The relationship between the size of the first bounding box BBX1 and the size of the second bounding box BBX2 may be, for example, the ratio of the vertical length of the first bounding box BBX1 to the vertical length of the second bounding box BBX2, or may be, for example, the ratio of the area of ​​the first bounding box BBX1 to the area of ​​the second bounding box BBX2. For example, as shown in Fig. 6, if the second bounding box BBX2 is located inside the first bounding box BBX1, there is a high correlation between the Euclidean distance d between the center coordinates C1 of the first bounding box BBX1 and the center coordinates C2 of the second bounding box BBX2 and the relationship between the size of the first bounding box BBX1 and the size of the second bounding box BBX2.

[0059] As a format for object detection results, a format including the size of the bounding box is relatively versatile, so the calculation of the positional relationship index value W can be simplified when the size of the bounding box is used, just as when the center coordinates of the bounding box are used.

[0060] When the camera shift detection device 3 notifies the surroundings that a position shift of the camera 1 has been detected, the method of notifying the surroundings is to send a message via the network to a pre-registered administrator of the object detection system 100. Alternatively, the camera shift detection device 3 may calculate the average value of the difference between the positional relationship index value W and the standard index value 323 over a certain period of time, estimate the installation condition of the camera 1 after the position shift (for example, the mounting angle of the camera 1 relative to the structure 4) from the calculated average value, and notify the estimated result as well. As another method of notifying the surroundings, the notification may be made by display or sound from a display or speaker attached to the object detection system 100.

[0061] Possible examples of using the estimation result of the installation status of camera 1 after positional displacement include the following: Object detection device 2 uses a learning data set using images taken in different environments (domains), with a different learning data set for each different environment (domain), and is configured to include multiple object detection models for each different environment (domain) that are constructed by artificial intelligence learning using the learning data set.

[0062] When the object detection device 2 receives the estimation result of the installation status of the camera 1 after the position shift from the camera misalignment detection device 3, the object detection device 2 then performs object detection using an object detection model corresponding to an environment (domain) close to the estimation result of the installation status of the camera 1 after the position shift. This makes it possible to suppress a decrease in the accuracy of object detection in the object detection device 2 even when the camera 1 is misaligned.

[0063] <3. Things to keep in mind> Various technical features disclosed in the description of the invention may be modified in various ways without departing from the spirit of the technical creation. Furthermore, the embodiments and modifications disclosed in the description of the invention may be combined to the extent possible.

[0064] For example, in the above-described embodiment, the object detected by the object detection device 2 is a person, the first specific part included in the object is the whole body of the person, and the second specific part included in the object is the head of the person, but the object detected by the object detection device 2 may be an object other than a person, such as a vehicle or a bicycle. The first specific part and the second specific part included in a vehicle may be, for example, a door mirror, a tire, a license plate, a headlight, etc. The first specific part and the second specific part included in a bicycle may be, for example, a front wheel, a rear wheel, a handlebar, a pedal, etc. For example, when the calculation unit 312 (controller 31) calculates a positional relationship index value related to a vehicle, the memory unit 32 may store the standard index value 323 corresponding to the object (vehicle). Also, for example, when the calculation unit 312 (controller 31) calculates a positional relationship index value related to a bicycle, the memory unit 32 may store the standard index value 323 corresponding to the object (bicycle). [Explanation of symbols]

[0065] 1. Camera 2. Object detection device 3. Camera displacement detection device 4...Structures 5...ground 6. Photographed images 31. Controller 32 Memory section 33. Communications Department 100 Object detection system

Claims

1. A method for detecting an installation anomaly when detecting a position shift of a camera, comprising the steps of: calculating a positional relationship index value indicating a positional relationship between a first specific location and a second specific location in an object image included in an image captured by the camera; comparing the calculated positional relationship index value with standard index values ​​for the first specific location and the second specific location set for the object in the object image; The method for detecting an installation anomaly detects a positional deviation of the camera based on a comparison result between the positional relationship index value and the standard index value.

2. The installation anomaly detection method according to claim 1 , further comprising the step of calculating the positional relationship index value using a correction coefficient according to a position of the object on the image.

3. the standard index value is a value specific to the object type of the object, calculating the positional relationship index value for objects of the same object type in a plurality of images captured by the camera; calculating an average index value by performing an averaging process on the calculated plurality of positional relationship index values; 2. The installation abnormality detection method according to claim 1, wherein the calculated average index value is regarded as the positional relationship index value and compared with the standard index value.

4. The installation abnormality detection method according to claim 1 , wherein the first specific location is a center position of a person's whole body, and the second specific location is a center position of the person's head.

5. the first specific location is an entire bounding box coordinate indicating a position of a bounding box of an entire object in the object image; the second specific part is a specific part bounding box coordinate indicating a position of a bounding box of the object specific part in the object image, The installation anomaly detection method according to claim 1 , wherein the positional relationship index value is a distance between the whole bounding box coordinates and the specific part bounding box coordinates.

6. The calculation of the positional relationship index value indicating a positional relationship on the image between a first specific location and a second specific location included in the same object includes: The installation anomaly detection method according to any one of claims 1 to 4, wherein the size of a bounding box of the entire object in the object image and the size of a bounding box of a specific part of the object in the object image are used for calculation.

7. The method is executed by a computer, and calculates a positional relationship index value indicating a positional relationship between a first specific location and a second specific location in an object image included in an image captured by the camera; comparing the calculated positional relationship index value with standard index values ​​for the first specific location and the second specific location set for the object in the object image; An installation abnormality detection program that detects a positional deviation of the camera based on a comparison result between the positional relationship index value and the standard index value.

8. calculating a positional relationship index value indicating a positional relationship between a first specific location and a second specific location in an object image included in an image captured by the camera; comparing the calculated positional relationship index value with standard index values ​​for the first specific location and the second specific location set for the object in the object image; An installation abnormality detection device that detects a positional deviation of the camera based on a comparison result between the positional relationship index value and the standard index value.

9. The installation abnormality detection device according to claim 8 , wherein when a positional deviation of the camera is detected, the detection of the positional deviation of the camera is notified to those nearby.

Images