Camera view calibration method, device and computer-readable storage medium

Abstract

The present application discloses a camera view angle calibration method, device, and computer-readable storage medium. The present application relates to the field of elevator technology. The camera view angle calibration method is applied to a calibration system. The method includes: obtaining monitoring image data collected in real time by a target camera in a target elevator car at preset time intervals, and determining a first open / closed state of a target car door of the target elevator car in the monitoring image data; when the first open / closed state is a partially open state or a fully open state, determining a monitoring open area of ​​the target car door in the monitoring image data; comparing the monitoring open area with a preset calibrated open area; and when the monitoring open area exceeds the calibrated open area, adjusting the monitoring view angle of the target camera based on the deviation between the monitoring open area and the calibrated open area. The present application can improve the effectiveness of camera view angle calibration.

Description

Technical Field

[0001] The present application relates to the field of elevator technology, and in particular to a camera perspective calibration method, device, and computer-readable storage medium. Background Art

[0002] In modern urban life, elevators are essential vertical transportation in high-rise buildings. Their safety and operational efficiency are directly related to residents' daily commuting and life safety. To ensure safety within elevator cabins, cameras are widely used for real-time monitoring, enabling rapid response and rescue measures in emergency situations. However, in actual use, due to frequent user activities such as moving and transporting goods, the position of cameras within elevator cabins can easily shift, causing the monitoring angle to deviate from the initially calibrated optimal angle, potentially failing to fully cover critical scenes within the elevator cabin.

[0003] Currently, there are two common methods for maintaining camera viewing angles. One is to manually use monitoring software to regularly or irregularly check the camera's viewing angle, and compare it with the initially calibrated optimal viewing angle to determine whether there is a deviation between the current viewing angle and the optimal viewing angle. This manual inspection method has the problems of high labor costs and low accuracy caused by manual image comparison. The other is an algorithm based on image brightness threshold or grayscale similarity, which calculates the difference between the monitoring image at the current viewing angle and the monitoring image at the calibrated monitoring angle to determine whether there is a deviation between the camera's current viewing angle and the optimal viewing angle. However, this method will be affected by environmental factors such as changes in light inside the elevator car, resulting in low accuracy of the calculated image difference.

[0004] Therefore, how to improve the effectiveness of camera perspective calibration is a problem that needs to be solved urgently. Summary of the Invention

[0005] The main purpose of this application is to provide a camera perspective calibration method, device and computer-readable storage medium, aiming to improve the effectiveness of camera perspective calibration.

[0006] To achieve the above objectives, the present application provides a camera view angle calibration method, which is applied to a calibration system and includes:

[0007] Acquire monitoring image data collected in real time by a target camera in a target elevator car at a preset time interval, and determine a first open / closed state of a target car door of the target elevator car in the monitoring image data;

[0008] When the first opening and closing state is a partially open state or a fully open state, determining a monitoring open area of ​​the target car door in the monitoring image data;

[0009] Comparing the monitored open area with a preset calibration open area, wherein the calibration open area is the open area of ​​the target car door in the calibration image data collected by the target camera when the target camera is in a calibration posture and the target car door is in a fully open state;

[0010] When the monitoring activation area exceeds the calibration activation area, the monitoring viewing angle of the target camera is adjusted based on the deviation between the monitoring activation area and the calibration activation area.

[0011] In one embodiment, the step of determining a first open / closed state of a target car door of the target elevator car in the monitoring image data comprises:

[0012] Inputting the monitoring image data into an image classification model to obtain a first open / closed state of a target car door of a target elevator car in the monitoring image data;

[0013] Among them, the image classification model is trained using the first image data collected by the camera in the elevator car as the model input data, and using the actual opening and closing state of the elevator car door in the first image data as the model training label.

[0014] In one embodiment, when the first opening and closing state is a partially open state or a fully open state, the step of determining the monitoring open area of ​​the target car door in the monitoring image data includes:

[0015] When the first opening and closing state is a partially open state or a fully open state, inputting the monitoring image data into an image segmentation model to obtain a monitoring open area of ​​the target car door in the monitoring image data;

[0016] Among them, the image segmentation model is trained using the second image data collected by the camera in the elevator car as model input data and the actual opening area of ​​the elevator car door in the second image data as the model training label. The second image data is collected by the camera when the car door is in an open state.

[0017] In one embodiment, before the step of comparing the monitoring activation area with a preset calibration activation area, the method further includes:

[0018] In response to a calibration detection request, obtaining calibration image data captured by the target camera when the target car door is in a fully open state, wherein the calibration detection request indicates that the target camera is in a calibration posture;

[0019] An opening area of ​​the target car door in the calibration image data is determined as a calibration opening area.

[0020] In one embodiment, the calibration system is in communication with an elevator control system, and the step of obtaining calibration image data captured by the target camera when the target car door is in a fully open state includes:

[0021] Acquiring third image data captured by the target camera, wherein the third image data is image data captured by the target camera when the calibration system receives a fully open signal sent by the elevator control system, the fully open signal indicating that the target car door is in a fully open state;

[0022] The third image data is determined as calibration image data collected by the target camera when the target car door is in a fully open state.

[0023] In one embodiment, the step of obtaining calibration image data collected by the target camera when the target car door is in a fully open state includes:

[0024] Acquiring video data collected by the target camera, wherein the video data includes at least one action of the target car door being fully opened;

[0025] Inputting the fourth image data of each frame in the video data into an image segmentation model respectively to obtain the first opening area of ​​the target car door in the fourth image data of each frame;

[0026] Calculate the area of ​​each of the first opening areas, and determine the fourth image data corresponding to the second opening area with the largest area among the first opening areas as the calibration image data collected by the target camera when the target car door is in a fully open state.

[0027] In one embodiment, the step of obtaining calibration image data collected by the target camera when the target car door is in a fully open state includes:

[0028] Obtaining fifth image data captured by the target camera, inputting the fifth image data into an image classification model, and obtaining a second open / closed state of the target car door in the fifth image data, wherein the fifth image data is image data captured by the target camera after the calibration system receives the calibration detection request;

[0029] When the second opening and closing state is a fully open state, the fifth image data is determined to be calibration image data collected by the target camera when the target car door is in a fully open state.

[0030] In one embodiment, after the step of determining the first open / closed state of the target car door of the target elevator car in the monitoring image data, the method further comprises:

[0031] In the case where the first opening and closing state is the closed state, the cumulative number is incremented by one, wherein the cumulative number is the number of times the target car door is continuously detected to be in the closed state;

[0032] When the accumulated number of times is greater than a preset number threshold, an early warning reminder is output to remind maintenance personnel to adjust the monitoring angle of the target camera in time.

[0033] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer-readable storage medium, and the computer-readable storage medium stores a program for implementing the camera perspective calibration method. The program for implementing the camera perspective calibration method is executed by a processor to implement the steps of the camera perspective calibration method as described above.

[0034] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, including a computer program, which implements the steps of the camera perspective calibration method as described above when executed by a processor.

[0035] The present application provides a camera view angle calibration method, which is applied to a calibration system. The present application obtains monitoring image data collected by a target camera in a target elevator car in real time once at a preset time interval, and determines the first opening and closing state of the target car door of the target elevator car in the monitoring image data. When the first opening and closing state is a partially open state or a fully open state, the opening area of ​​the target car door in the monitoring image data (i.e., the monitoring opening area) is determined, and then the monitoring opening area is compared with the preset calibration opening area, wherein the calibration opening area is the opening area of ​​the target car door in the calibration image data collected by the target camera when the target camera is in a calibration posture and the target car door is in a fully open state. Finally, when it is detected that the monitoring opening area exceeds the calibration opening area, the monitoring view angle of the target camera is adjusted based on the deviation between the monitoring opening area and the calibration opening area.

[0036] In summary, the present application obtains the monitoring image data collected by the target camera once every preset time interval through the calibration system, and identifies the opening and closing status of the target car door in the monitoring image data. When it is determined that the target car door is in the open state at this time, the monitoring opening area of ​​the target car door in the monitoring image data is determined, and the monitoring opening area is compared with the preset calibration opening area. It should be understood that the calibration opening area is the maximum range of the target car door opening area when the target camera is in the calibration posture. Therefore, the opening area of ​​the target car door is used as a reference to determine whether the target camera has a viewing angle offset, that is, when it is detected that the monitoring opening area exceeds the calibration opening area, it is determined that the target camera has a viewing angle offset, and the monitoring viewing angle is automatically adjusted based on the deviation between the monitoring opening area and the calibration opening area. In this way, in this application, the process of judging and calibrating the camera perspective offset does not require human intervention, and compared to the method of misjudging the perspective deviation due to changes in light in the elevator car, this application uses the open area of ​​the elevator car door as a reference for judging the perspective deviation, avoiding the influence of other areas in the image data on the final judgment result, thereby improving the accuracy of the camera perspective offset judgment, and further improving the effectiveness of calibrating the camera perspective based on the camera perspective offset judgment result. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0039] Figure 1 This is a flow chart of the first embodiment of the camera view angle calibration method of the present application;

[0040] Figure 2 Schematic diagram of a camera view angle calibration architecture according to an embodiment of the camera view angle calibration method of the present application;

[0041] Figure 3 This is a schematic diagram of a first ROI determination process involved in an embodiment of the camera view angle calibration method of the present application;

[0042] Figure 4 Schematic diagram of a second ROI determination process involved in an embodiment of the camera view angle calibration method of the present application;

[0043] Figure 5Schematic diagram of a third ROI determination process involved in an embodiment of the camera view angle calibration method of the present application;

[0044] Figure 6 A schematic diagram of a camera view angle calibration process according to an embodiment of the camera view angle calibration method of the present application;

[0045] Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the camera perspective calibration method in the embodiment of the present application.

[0046] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0048] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0049] The main solution of the present application is: obtaining monitoring image data collected in real time by a target camera in a target elevator car at preset time intervals, and determining a first opening and closing state of a target car door of the target elevator car in the monitoring image data; when the first opening and closing state is a partially open state or a fully open state, determining a monitoring opening area of ​​the target car door in the monitoring image data; comparing the monitoring opening area with a preset calibration opening area, wherein the calibration opening area is the opening area of ​​the target car door in the calibration image data collected by the target camera when the target camera is in a calibration posture and the target car door is in a fully open state; when the monitoring opening area exceeds the calibration opening area, adjusting the monitoring viewing angle of the target camera based on the deviation between the monitoring opening area and the calibration opening area.

[0050] Currently, there are two common methods for maintaining camera viewing angles. One is to manually use monitoring software to periodically or irregularly check the camera's viewing angle and compare it with the initially calibrated optimal viewing angle to determine whether the current viewing angle deviates from the optimal one. This manual inspection method is labor-intensive and has low accuracy due to manual image comparison. It also fails to detect camera viewing angle deviations in a timely manner, potentially leading to missed detections. The other method uses motion detection or image similarity to calculate the difference between the current viewing angle and the optimal viewing angle. Motion detection and image similarity methods also face challenges in practical applications. The light source inside the elevator car is usually not on all day, and lighting conditions vary significantly between day and night. This change in light intensity can cause significant changes in image brightness and contrast, which can cause algorithms based on image brightness thresholds or grayscale similarity to misjudge. This is especially true when the light level is low. There may be strong light sources or shadowed areas inside the elevator car. This uneven light distribution also affects the accuracy of motion detection algorithms, as the algorithms have difficulty distinguishing between pixel changes caused by actual motion and light fluctuations. If the interior of the elevator car is made of reflective materials, such as metal or reflective glass, these surfaces will reflect surrounding light, causing the camera to capture areas of strong reflection. These reflections may be mistaken for movement or overlap with actual movement, thereby interfering with the accuracy of the motion detection algorithm. This can lead to misjudgment and omission of camera angle changes, resulting in low accuracy in the calculated image difference. Therefore, more reliable and accurate methods are needed to monitor deviations in camera angles to ensure the completeness and effectiveness of monitoring coverage.

[0051] The present application obtains the monitoring image data collected by the target camera once every preset time interval through the calibration system, and identifies the opening and closing status of the target car door in the monitoring image data. When it is determined that the target car door is in the open state at this time, the monitoring opening area of ​​the target car door in the monitoring image data is determined, and the monitoring opening area is compared with the preset calibration opening area. It should be understood that the calibration opening area is the maximum range of the target car door opening area when the target camera is in the calibration posture. Therefore, the opening area of ​​the target car door is used as a reference to determine whether the target camera has a viewing angle offset, that is, when it is detected that the monitoring opening area exceeds the calibration opening area, it is determined that the target camera has a viewing angle offset, and the monitoring viewing angle is automatically adjusted based on the deviation between the monitoring opening area and the calibration opening area. As such, in this application, the process of judging and calibrating the camera perspective offset does not require manual intervention, reducing operating and maintenance costs. Compared to the method of misjudging the perspective deviation due to changes in light in the elevator car, this application uses the open area of ​​the elevator car door as a reference for judging the perspective deviation, avoiding the influence of other areas in the image data on the final judgment result, thereby improving the accuracy of the camera perspective offset judgment, and further improving the effectiveness of calibrating the camera perspective based on the camera perspective offset judgment result. In addition, by effectively maintaining the perspective of the surveillance camera, the present application can monitor the situation inside the elevator in real time, avoid monitoring blind spots and misjudgments, and effectively prevent and record any potential safety issues or dangerous behaviors, such as people being trapped or the occurrence of blind spots in monitoring. This not only helps to protect the personal safety of passengers, but also improves the management efficiency of elevator operations and ensures the safe use of passengers and facilities.

[0052] It should be noted that the execution entity of the camera view angle calibration method in each embodiment of the present application can be a calibration system, or a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or a camera view angle calibration device capable of performing the above functions, etc., and this embodiment does not specifically limit this. The following uses the calibration system as the execution entity as an example to illustrate this embodiment and the following embodiments.

[0053] Based on this, this application proposes a camera viewing angle calibration method of the first embodiment, please refer to Figure 1 The camera viewing angle calibration method includes steps S10 to S40:

[0054] Step S10, acquiring monitoring image data collected in real time by a target camera in a target elevator car at a preset time interval, and determining a first open / closed state of a target car door of the target elevator car in the monitoring image data;

[0055] It should be noted that the cycle length for executing the camera viewing angle inspection is pre-set, that is, the above-mentioned preset duration.

[0056] The image data (hereinafter referred to as monitoring image data for distinction) including the interior of the elevator car, collected in real time by a camera (hereinafter referred to as target camera for distinction) installed in the elevator car (hereinafter referred to as target elevator car for distinction) is obtained once at a preset time interval, and then the opening and closing state (hereinafter referred to as first opening and closing state for distinction) of the car door of the target elevator car (hereinafter referred to as target car door for distinction) in the monitoring image data is determined. Among them, the opening and closing state of the car door is divided into three categories, namely, fully open state, partially open state and closed state, and it should be understood that when the opening and closing state of the car door is fully open state or partially open state, the opening and closing state of the car door is considered to be open state.

[0057] In one feasible implementation, assuming the preset time duration is 10 minutes and the calibration system's most recent camera angle check operation was at a first check time, the calibration system needs to perform the next camera angle check operation at a second check time, where the second check time is 10 minutes apart from the first check time and the second check time is later than the first check time. Specifically, monitoring image data captured by the target camera is obtained at the first check time. It should be understood that the monitoring image data is the image data captured by the target camera at the first check time.

[0058] In this embodiment, step S10 may include:

[0059] Step S101: inputting the monitoring image data into an image classification model to obtain a first open / closed state of a target car door of a target elevator car in the monitoring image data;

[0060] Among them, the image classification model is trained using the first image data collected by the camera in the elevator car as the model input data, and using the actual opening and closing state of the elevator car door in the first image data as the model training label.

[0061] It should be noted that the image data of the interior of each elevator car collected by the cameras installed in various types of elevator cars (hereinafter referred to as the first image data for distinction) are obtained in advance, and when the camera collects the first image data, the elevator car door can be in a closed state, a partially open state, and a fully open state, and the actual opening and closing state of the elevator car door when the camera collects each first image data (i.e., the above-mentioned real opening and closing state) is determined, wherein the real opening and closing state is divided into three categories, namely, a fully open state, a partially open state, and a closed state. A first image data and the real opening and closing state corresponding to the first image data are used as a training sample, and a training sample set is formed based on multiple such training samples. When training the initial image classification model, the first image data in each training sample in the training sample set is used as the model input data, and the real opening and closing state in each training sample is used as the model training label. The initial image classification model to be trained is trained to obtain the above-mentioned image classification model.

[0062] The image classification model obtained after training has the function of identifying the opening and closing status of the car door in the image data. Therefore, when the calibration system performs the camera view inspection operation, the monitoring image data obtained at the inspection time is input into the image classification model, and the monitoring image data is analyzed by the image classification model to output the first opening and closing status of the target car door in the monitoring image data.

[0063] Step S20, when the first opening and closing state is a partially open state or a fully open state, determining a monitoring open area of ​​the target car door in the monitoring image data;

[0064] It should be noted that when the first opening and closing state output by the image classification model is a fully open state or a partially open state, the first opening and closing state is considered to be an open state. In addition, the open area of ​​the car door refers to the circumscribed rectangular box of the open area of ​​the car door in the image data containing the interior of the elevator car.

[0065] When the first opening and closing state output by the image classification model is a fully open state or a partially open state, that is, when the target car door is recognized to be open, the opening area of ​​the target car door in the monitoring image data is determined (hereinafter referred to as the monitoring opening area for distinction).

[0066] In this embodiment, step S20 may include:

[0067] Step S201: when the first opening and closing state is a partially open state or a fully open state, inputting the monitoring image data into an image segmentation model to obtain a monitoring open area of ​​the target car door in the monitoring image data;

[0068] Among them, the image segmentation model is trained using the second image data collected by the camera in the elevator car as model input data and the actual opening area of ​​the elevator car door in the second image data as the model training label. The second image data is collected by the camera when the car door is in an open state.

[0069] It should be noted that the image data of the interior of each elevator car collected by the cameras installed in various types of elevator cars (hereinafter referred to as the second image data for distinction) are obtained in advance, and when the camera collects the second image data, the elevator car door can be in a partially open state and a fully open state, that is, to ensure that the elevator car door is open. Then, the actual opening area of ​​the elevator car door in each second image data is determined (hereinafter referred to as the real opening area for distinction). Similarly, a second image data and the real opening area corresponding to the second image data are used as a training sample, and a training sample set is formed based on multiple such training samples. When training the initial image segmentation model, the second image data in each training sample in the training sample set is used as the model input data, and the real opening area in each training sample is used as the model training label to train the initial image segmentation model to be trained, so as to obtain the above-mentioned image segmentation model.

[0070] The image segmentation model obtained after training has the function of identifying the open and closed areas of the car door in the image data. Therefore, when the calibration system performs the camera view inspection operation, if the target car door in the monitoring image data obtained at the time of inspection is in an open state, then the monitoring image data is further input into the image segmentation model, and the monitoring image data is analyzed by the image segmentation model to output the monitoring open area of ​​the target car door in the monitoring image data. It should be understood that the open area of ​​the car door can be a circumscribed rectangular box containing the door open area in the image data of the interior of the car. Determining the open area is to determine the position information (i.e., coordinates) of the circumscribed rectangular box of the door open area in the image data.

[0071] Step S30, comparing the monitored open area with a preset calibration open area, wherein the calibration open area is the open area of ​​the target car door in the calibration image data collected by the target camera when the target camera is in a calibration posture and the target car door is in a fully open state;

[0072] It should be noted that when the target camera is first installed in the target elevator car, the installer usually debugs and calibrates the position of the target camera until the target camera is in the calibration posture. Since the embodiment of the present application uses the opening area of ​​the car door as a reference for judging whether the camera viewing angle is offset, when the target camera is in the calibration posture and the target car door is in the fully open state, the opening area of ​​the target car door (hereinafter referred to as the calibration image data for distinction) in the image data collected by the target camera is in the calibration posture (hereinafter referred to as the calibration image data for distinction). It can be understood that the calibration opening area is also called the reference ROI (region of interest).

[0073] After determining the calibration start-up area, the monitoring start-up area is compared with the calibration start-up area. It should be understood that the size of the image data collected by each camera is the same. Therefore, in a specific embodiment, the coordinate position of the monitoring start-up area in the monitoring image data and the coordinate position of the calibration start-up area in the calibration image data are compared to determine whether the target camera viewing angle is offset.

[0074] Step S40 : When the monitoring activation area exceeds the calibration activation area, adjusting the monitoring viewing angle of the target camera based on the deviation between the monitoring activation area and the calibration activation area.

[0075] When it is detected that the monitoring start-up area exceeds the calibration start-up area, the monitoring viewing angle of the target camera is reversely adjusted based on the position deviation between the monitoring start-up area and the calibration start-up area.

[0076] In a specific embodiment, the calibration system can remotely control the posture of the target camera. When the monitoring activation area exceeds the calibration activation area, the difference between the position coordinates of the same corner point in the monitoring activation area and the calibration activation area is calculated, and the posture of the target camera is adjusted based on this difference to change its monitoring angle of view, thereby ensuring that the target camera can effectively cover the key scenes in the target elevator car.

[0077] In an embodiment of the present application, the calibration system obtains the monitoring image data collected by the target camera once every preset time interval, and identifies the opening and closing status of the target car door in the monitoring image data. When it is determined that the target car door is in the open state at this time, the monitoring opening area of ​​the target car door in the monitoring image data is determined, and the monitoring opening area is compared with the preset calibration opening area. It should be understood that the calibration opening area is the maximum range of the target car door opening area when the target camera is in the calibration posture. Therefore, the opening area of ​​the target car door is used as a reference to determine whether the target camera has a viewing angle deviation, that is, when it is detected that the monitoring opening area exceeds the calibration opening area, it is determined that the target camera has a viewing angle deviation, and the monitoring viewing angle is automatically adjusted based on the deviation between the monitoring opening area and the calibration opening area. As such, in this application, the process of judging and calibrating the camera perspective offset does not require manual intervention, reducing operating and maintenance costs. Compared to the method of misjudging the perspective deviation due to changes in light in the elevator car, this application uses the open area of ​​the elevator car door as a reference for judging the perspective deviation, avoiding the influence of other areas in the image data on the final judgment result, thereby improving the accuracy of the camera perspective offset judgment, and further improving the effectiveness of calibrating the camera perspective based on the camera perspective offset judgment result. In addition, by effectively maintaining the perspective of the surveillance camera, the present application can monitor the situation inside the elevator in real time, avoid monitoring blind spots and misjudgments, and effectively prevent and record any potential safety issues or dangerous behaviors, such as people being trapped or the occurrence of blind spots in monitoring. This not only helps to protect the personal safety of passengers, but also improves the management efficiency of elevator operations and ensures the safe use of passengers and facilities.

[0078] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above introduction and will not be repeated hereafter. On this basis, before the step S30, the camera angle calibration method further includes:

[0079] Step A10, in response to a calibration detection request, obtaining calibration image data collected by the target camera when the target car door is in a fully open state, wherein the calibration detection request indicates that the target camera is in a calibration posture;

[0080] It should be noted that when the target camera is installed in the target elevator car for the first time, the installer usually installs the target camera on the top left or right side of the target elevator car, and then debugs and calibrates the position of the target camera until the target camera is in the calibration posture, wherein the calibration posture can ensure that the target camera's viewing angle range can include the elevator car floor and the elevator car door. In an embodiment of the present application, when the target camera is installed in place, that is, when the target camera is in the calibration posture, the calibration start area is determined. Specifically, the calibration system may include a user operation interface or a physical button. When the installer determines that the target camera is in the calibration posture and there is a need to determine the calibration start area, a calibration detection request is triggered based on the user operation interface or the physical button to instruct the calibration system to obtain calibration image data to further determine the calibration start area.

[0081] After receiving the calibration detection request, the calibration system responds to the calibration detection request and obtains calibration image data collected by the target camera when the target car door is in a fully open state.

[0082] In this embodiment, the calibration system is in communication with the elevator control system, and step A10 may include:

[0083] Step A101, obtaining third image data captured by the target camera, wherein the third image data is image data captured by the target camera when the calibration system receives a fully open signal sent by the elevator control system, and the fully open signal indicates that the target car door is in a fully open state;

[0084] It should be noted that if Figure 2 The figure shows a schematic diagram of the camera view calibration architecture. The calibration system is connected to the elevator control system, and the calibration system can control the camera's posture based on the camera view inspection results. The elevator control system can monitor the open and closed status of the elevator car door. When the elevator control system detects that the target car door is fully open, it is configured to send a signal (the fully open signal) to the calibration system to inform the calibration system that the target car door is fully open.

[0085] When the full-open signal sent by the elevator control system is received, the image data collected by the target camera at the same time is obtained (hereinafter referred to as the third image data for distinction).

[0086] Step A102: Determine the third image data as calibration image data collected by the target camera when the target car door is in a fully open state.

[0087] It should be understood that, without considering the delay in signal transmission, it can be assumed that the target car door is fully open when the calibration system receives the full-open signal, so the third image data collected by the target camera when the calibration system receives the full-open signal is the calibration image data.

[0088] In this embodiment, step A10 may include:

[0089] Step A103, obtaining video data collected by the target camera, wherein the video data includes at least one action of the target car door being fully opened;

[0090] It should be noted that manual intervention can be used to ensure that a certain section of the elevator's operation includes at least one complete door opening action, and record the video data collected by the target camera during this section of operation. The user then triggers a calibration detection request based on the user operation interface or physical buttons provided by the calibration system, and the calibration detection request includes the storage location of the video data.

[0091] After receiving the calibration detection request, the calibration system parses the calibration detection request and obtains video data including at least one action of the target car door being fully opened from a storage location specified by the calibration detection request.

[0092] Step A104: inputting the fourth image data of each frame in the video data into an image segmentation model to obtain the first opening area of ​​the target car door in the fourth image data of each frame;

[0093] The video data is decomposed to obtain each frame of image data (hereinafter referred to as the fourth image data for distinction), and each frame of the fourth image data is input into a pre-trained image segmentation model to obtain the opening area of ​​the target car door in each frame of the fourth image data (hereinafter referred to as the first opening area for distinction).

[0094] Step A105, calculate the area of ​​each of the first opening areas, and determine the fourth image data corresponding to the second opening area with the largest area among the first opening areas as the calibration image data collected by the target camera when the target car door is in a fully open state.

[0095] After obtaining the first opening area of ​​the target car door in each frame of the fourth image data, the area of ​​each first opening area is calculated based on the coordinate position of each first opening area, and the fourth image data corresponding to the opening area with the largest area among the first opening areas (hereinafter referred to as the second opening area for distinction) is determined as the calibration image data. It should be understood that since the video data contains the action of the target car door being fully opened and the target camera is in the calibration posture during the video data recording process, that is, the monitoring angle of the target camera includes the car floor and the car door, it can be considered that the car door is fully opened when the opening area of ​​the car door is the largest.

[0096] In this embodiment, step A10 may include:

[0097] Step A106: Acquire fifth image data captured by the target camera, input the fifth image data into an image classification model, and obtain the second open / closed state of the target car door in the fifth image data, wherein the fifth image data is image data captured by the target camera after the calibration system receives the calibration detection request;

[0098] After receiving the calibration detection request, the calibration system collects image data in real time by the target camera (hereinafter referred to as the fifth image data for distinction), inputs the fifth image data collected in real time into the image classification model, analyzes the fifth image data through the image classification model, and outputs the opening and closing status of the target elevator door in the fifth image data (hereinafter referred to as the second opening and closing status for distinction), where the second opening and closing status is a fully open state, a partially open state or a closed state.

[0099] It should be noted that the present application does not limit the frequency of acquiring the fifth image data captured by the target camera. For example, in one feasible embodiment, the calibration system acquires the fifth image data captured by the target camera once every one minute, and analyzes the fifth image data through the image classification model to obtain the open and closed state of the target car door at the time of acquisition of the fifth image data. For another example, in another feasible embodiment, after receiving the calibration detection request, the calibration system acquires the fifth image data captured by the target camera at each moment, and classifies the fifth image data acquired in real time through the image classification model to obtain the open and closed state of the target car door at each moment, until the target car door is detected to be fully open.

[0100] Step A107: When the second opening and closing state is a fully open state, determine that the fifth image data is calibration image data collected by the target camera when the target car door is in a fully open state.

[0101] When the second opening and closing state is the fully open state, the current fifth image data is determined to be the calibration image data. It should be understood that the installer triggers the calibration detection request when the camera is in the calibration posture. After receiving the calibration detection request, the calibration system obtains the fifth image data. It can be considered that the camera will not change its position for a long time after it is installed in place, so the fifth image data representing the fully open target car door is determined to be the calibration image data.

[0102] Step A20: determining the opening area of ​​the target car door in the calibration image data as the calibration opening area.

[0103] After the calibration image data is determined, the opening area of ​​the target car door in the calibration image data is determined as the calibration opening area.

[0104] In a first feasible implementation, after receiving a calibration detection request triggered by a user, it is monitored whether a full-open signal sent by the elevator control system is received. If a full-open signal is received, the third image data collected by the target camera when the calibration system receives the full-open signal sent by the elevator control system is obtained, and the third image data is determined as calibration image data. Then, the calibration image data is input into a pre-trained image classification model to obtain a calibration open area in the calibration image data.

[0105] In a second feasible implementation manner, after receiving a calibration detection request triggered by the user, video data including at least one action of the target car door being fully opened is obtained based on the storage location represented by the calibration detection request, and the fourth image data of each frame in the video data is respectively input into the image segmentation model to obtain the first opening area of ​​the target car door in the fourth image data of each frame, and the fourth image data corresponding to the second opening area with the largest area in each first opening area is determined as the calibration image data, and the second opening area is determined as the calibration opening area.

[0106] In a third feasible implementation, after receiving a calibration detection request triggered by the user, the fifth image data captured by the target camera is acquired in real time, and the second opening and closing state of the target car door in the fifth image data is determined based on the image classification model, until the second opening and closing state is a fully open state, then the fifth image data currently representing the fully open state is determined to be the calibration image data, and then the calibration image data is input into the image segmentation model to obtain the calibrated opening area of ​​the target car door in the calibration image data.

[0107] For example, the installer installs the camera in the calibration posture, ensures that the camera's field of view includes the elevator car floor and the elevator car door, and then triggers a calibration detection request; after receiving the calibration detection request, the calibration system responds to the calibration detection request and obtains calibration image data, wherein the calibration image data can be obtained in three ways, such as Figure 3 The figure shows a schematic diagram of the first ROI determination process. The first method is: detecting whether the full-open signal sent by the elevator control system is received. If the full-open signal is received, obtaining the third image data collected by the camera when the full-open signal is received, and using the third image data as the calibration image data; Figure 4 FIG. 1 is a schematic diagram of a second ROI determination process. The second method is as follows: parsing the storage address pointed to by the calibration detection request, obtaining video data from the storage address, the video data including at least one action of the target car door being fully opened, inputting the fourth image data of each frame in the video data into the image segmentation model, obtaining the open area of ​​the car door in each fourth image data, and determining the fourth image data corresponding to the open area with the largest area among the open areas as the calibration image data; Figure 5 FIG2 is a schematic diagram of the third ROI determination process. The third method is: obtaining the fifth image data collected by the camera in real time, and identifying the open and closed state of the car door in the fifth image data based on the image classification model. When it is identified that the car door is fully open, the current fifth image data is determined to be the calibration image data; finally, the calibration open area in the calibration image data is determined.

[0108] In this way, since the embodiment of the present application uses the opening area of ​​the car door in the image data collected by the camera when the camera is in the calibration posture and the car door is in the fully open state as the reference area (i.e., the above-mentioned calibration opening area), the monitoring opening area of ​​the car door in the monitoring image data obtained at the time of the camera angle inspection is compared with the calibration opening area. When the monitoring opening area is within the range of the calibration opening area, it is determined that the camera angle of view is normal. Otherwise, it is determined that the camera angle of view is offset and the angle of view needs to be adjusted. It can be seen that the determination of the calibration opening area is intuitive and important. In the embodiment of the present application, three methods of determining the calibration opening area are provided to ensure the effective determination of whether the subsequent camera angle of view is offset.

[0109] Based on the first embodiment and / or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to above and will not be described in detail. On this basis, after step S10, the camera angle calibration method further includes:

[0110] Step B10: when the first opening and closing state is the closed state, adding one to the cumulative number of times, wherein the cumulative number of times the target car door is continuously detected to be in the closed state;

[0111] Step B20: When the accumulated number of times is greater than a preset number threshold, an early warning reminder is output to remind maintenance personnel to adjust the monitoring angle of the target camera in time.

[0112] It should be noted that since the calibration system performs a camera angle check at preset intervals, the results of each camera angle check are recorded, primarily the number of consecutive detections of the car door closing. This means that each time the car door is detected to be open, the cumulative count is reset to zero, and each time the car door is detected to be closed, the cumulative count is incremented. An upper threshold for the cumulative count, i.e., the preset threshold, is also pre-set.

[0113] When it is detected that the first opening and closing state is the closed state, that is, when the target car door is in the closed state, the cumulative number is increased by one, and it is detected whether the cumulative number is greater than the preset number. If it is detected that the cumulative number is greater than the preset number, it is considered that there is no car door under the monitoring angle of the target camera, that is, the target camera may have a large position offset, so the calibration system issues an early warning reminder to prompt the maintenance personnel to adjust the position of the target camera in time.

[0114] For example, Figure 6 The figure shows a schematic diagram of the camera view calibration process, which obtains the monitoring image data collected by the camera at a preset time interval; determines whether the car door in the monitoring image data is open; if the car door is detected to be open, determines the opening area of ​​the car door in the monitoring image data (i.e., the monitoring opening area); then, determines whether the monitoring opening area exceeds the preset calibrated opening area. If so, determines that the camera view is offset, and adjusts the camera posture based on the difference between the monitoring opening area and the calibrated opening area. If not, determines that the camera view has not offset; if the car door is not detected to be open, determines whether the cumulative number of consecutive times the car door is not detected to be open is greater than the preset number threshold. When the cumulative number is greater than the preset number threshold, determines that the camera view has changed.

[0115] In this way, the embodiment of the present application can identify the special situation where the camera posture has a large deviation, resulting in the car door being out of the camera's field of view. When a large deviation of the camera is detected, it is difficult to calibrate the camera's field of view by automatically controlling the camera to adjust its posture through the calibration system. Therefore, the calibration system needs to output an early warning reminder at this time to request manual intervention in the calibration, thereby improving the stability of the calibration system.

[0116] The present application provides a camera perspective calibration device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the camera perspective calibration method in the above-mentioned embodiment one.

[0117] Reference below Figure 7 , which shows a schematic structural diagram of a camera view angle calibration device suitable for implementing embodiments of the present application. The camera view angle calibration device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Displays), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 7 The camera perspective calibration device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0118] like Figure 7As shown, the camera perspective calibration device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read On ly Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. Various programs and data required for the operation of the camera perspective calibration device are also stored in RAM 1004. The processing device 1001, ROM 1002, and RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the camera perspective calibration device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a camera perspective calibration device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or have instead.

[0119] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0120] The camera perspective calibration device provided in this application, employing the camera perspective calibration method described in the aforementioned embodiment, can address the technical problem of improving the effectiveness of camera perspective calibration. Compared to the prior art, the camera perspective calibration device provided in this application achieves the same beneficial effects as the camera perspective calibration method described in the aforementioned embodiment. Other technical features of the camera perspective calibration device are the same as those disclosed in the aforementioned embodiment and are not further elaborated upon here.

[0121] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0122] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0123] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, and the computer-readable program instructions are used to execute the camera view calibration method in the above-mentioned embodiment.

[0124] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0125] The computer-readable storage medium may be included in the camera view calibration device; or it may exist independently without being assembled into the camera view calibration device.

[0126] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the camera view calibration device, the camera view calibration device is enabled to: obtain the monitoring image data collected by the target camera in the target elevator car in real time at a preset time interval, and determine the first opening and closing state of the target car door of the target elevator car in the monitoring image data; when the first opening and closing state is a partially open state or a fully open state, determine the monitoring opening area of ​​the target car door in the monitoring image data; compare the monitoring opening area with a preset calibration opening area, wherein the calibration opening area is the opening area of ​​the target car door in the calibration image data collected by the target camera when the target camera is in a calibration posture and the target car door is in a fully open state; when the monitoring opening area exceeds the calibration opening area, adjust the monitoring view of the target camera based on the deviation between the monitoring opening area and the calibration opening area.

[0127] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0128] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0129] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0130] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned camera view angle calibration method. This computer-readable storage medium can address the technical problem of improving the effectiveness of camera view angle calibration. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the camera view angle calibration method provided in the aforementioned embodiment, and are not further elaborated here.

[0131] An embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the camera view angle calibration method as described above when executed by a processor.

[0132] The computer program product provided in this application can improve the effectiveness of camera perspective calibration. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiments of this application are the same as the beneficial effects of the camera perspective calibration method provided in the above embodiments, and will not be repeated here.

[0133] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A camera viewing angle calibration method, characterized in that: The camera view angle calibration method is applied to a calibration system, and the camera view angle calibration method includes: Acquire monitoring image data collected in real time by a target camera in a target elevator car at a preset time interval, and determine a first open / closed state of a target car door of the target elevator car in the monitoring image data; When the first opening and closing state is a partially open state or a fully open state, inputting the monitoring image data into an image segmentation model to obtain a monitoring open area of ​​the target car door in the monitoring image data; The image segmentation model is trained using second image data captured by a camera inside the elevator car as model input data, and using the actual open area of ​​the elevator car door in the second image data as a model training label, wherein the second image data is captured by the camera when the car door is in an open state; Comparing the monitored open area with a preset calibration open area, wherein the calibration open area is the open area of ​​the target car door in the calibration image data collected by the target camera when the target camera is in a calibration posture and the target car door is in a fully open state; When the monitoring activation area exceeds the calibration activation area, adjusting the monitoring viewing angle of the target camera based on the deviation between the monitoring activation area and the calibration activation area; The step of determining a first open / closed state of a target car door of the target elevator car in the monitoring image data comprises: Inputting the monitoring image data into an image classification model to obtain a first open / closed state of a target car door of a target elevator car in the monitoring image data; The image classification model is trained using first image data captured by a camera in an elevator car as model input data, and using the actual open and closed state of the elevator car door in the first image data as model training labels; Before the step of comparing the monitoring activation area with the preset calibration activation area, the method further includes: In response to a calibration detection request, obtaining calibration image data captured by the target camera when the target car door is in a fully open state, wherein the calibration detection request indicates that the target camera is in a calibration posture; An opening area of ​​the target car door in the calibration image data is determined as a calibration opening area.

2. The method according to claim 1, wherein The calibration system is in communication with the elevator control system, and the step of obtaining calibration image data collected by the target camera when the target car door is in a fully open state includes: Acquiring third image data captured by the target camera, wherein the third image data is image data captured by the target camera when the calibration system receives a fully open signal sent by the elevator control system, the fully open signal indicating that the target car door is in a fully open state; The third image data is determined as calibration image data collected by the target camera when the target car door is in a fully open state.

3. The method according to claim 1, wherein The step of obtaining calibration image data collected by the target camera when the target car door is in a fully open state includes: Acquiring video data collected by the target camera, wherein the video data includes at least one action of the target car door being fully opened; Inputting the fourth image data of each frame in the video data into an image segmentation model respectively to obtain the first opening area of ​​the target car door in the fourth image data of each frame; Calculate the area of ​​each of the first opening areas, and determine the fourth image data corresponding to the second opening area with the largest area among the first opening areas as the calibration image data collected by the target camera when the target car door is in a fully open state.

4. The method according to claim 1, wherein The step of obtaining calibration image data collected by the target camera when the target car door is in a fully open state includes: Obtaining fifth image data captured by the target camera, inputting the fifth image data into an image classification model, and obtaining a second open / closed state of the target car door in the fifth image data, wherein the fifth image data is image data captured by the target camera after the calibration system receives the calibration detection request; When the second opening and closing state is a fully open state, the fifth image data is determined to be calibration image data collected by the target camera when the target car door is in a fully open state.

5. The method according to any one of claims 1 to 4, characterized in that After the step of determining the first open / closed state of the target car door of the target elevator car in the monitoring image data, the method further includes: In the case where the first opening and closing state is the closed state, the cumulative number is incremented by one, wherein the cumulative number is the number of times the target car door is continuously detected to be in the closed state; When the accumulated number of times is greater than a preset number threshold, an early warning reminder is output to remind maintenance personnel to adjust the monitoring angle of the target camera in time.

6. A camera viewing angle calibration device, characterized in that: The camera view angle calibration device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the camera view angle calibration method according to any one of claims 1 to 5.

7. A storage medium, characterized in that: The storage medium is a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the camera view angle calibration method according to any one of claims 1 to 5 are implemented.

Images