A kind of visual inspection system, method and storage medium based on intelligent camera

Abstract

The application relates to the technical field of image recognition, in particular to a visual detection system and method based on an intelligent camera and a storage medium. The method comprises the following steps: in step S1, a first image of a passenger entering an elevator and a shooting time are acquired through an internal first camera, and face information of the passenger is acquired based on the first image; in step S2, a predicted floor of the passenger getting off the elevator is selected according to an image of the passenger, when the predicted floor of the passenger getting off the elevator cannot be acquired, historical boarding information is searched in a storage unit, and the predicted floor of the passenger getting off the elevator is acquired; in step S3, predicted floors of all passengers in the elevator are acquired; in step S4, a first residual space in the elevator is acquired; a demand space of a floor passenger is calculated through a third image, and whether the elevator is stopped at the floor is determined according to the first residual space or a second residual space and the demand space. The application solves the problem of low elevator operation efficiency and improves the operation efficiency of the elevator.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, more particularly to a visual detection system and method based on an intelligent camera and a storage medium. BACKGROUND

[0002] Elevator is the main traffic equipment in modern buildings. At present, there are high-efficiency and mature solutions in the dispatching of elevators. Under this condition, the efficiency of elevator stopping, passenger getting off and getting on is the main factor affecting the efficiency of elevator operation. In the prior art, for example, Chinese patent CN110654946B discloses a community elevator dispatching method and system based on artificial intelligence. The method comprises: acquiring a call request, a destination floor of each elevator, a load state of the elevator and predicted floor information; establishing an artificial intelligence optimization model according to the call request, the destination floor of each elevator, the load state of the elevator and the predicted floor information; the artificial intelligence optimization model adopts a genetic algorithm to generate an implementation dispatching scheme. The method updates the dispatching scheme in real time, selects the optimal real-time dispatching scheme, so that the elevator can operate efficiently and improve the use efficiency of the elevator. Also for example, CN113247718B, the embodiment of the present application relates to the field of communication technology, and discloses an elevator dispatching method, system, cloud server, electronic device and storage medium. In the present application, the elevator dispatching method is applied to the cloud server in the elevator dispatching system, and the method comprises: acquiring image data photographed by an in-elevator camera and an out-elevator camera arranged on each floor; generating an elevator dispatching instruction according to the image data; and sending the elevator dispatching instruction to a target elevator through a network. Through the above means, different elevators can be uniformly dispatched through the network server, the operation efficiency of the elevator is improved, and the waiting time of the user is reduced. The above two patents do not consider the problem that the elevator will stop at each floor when the load of the elevator is less than the rated load but there is no space in the elevator, do not consider the problem that the area is divided according to the passenger's predicted floor station in the elevator, do not consider the problem that the passenger's moving speed is detected according to the image when the passenger gets off the elevator, the time when the last passenger leaves the elevator door is calculated, and the elevator door can be closed when the last passenger leaves the elevator, and do not consider the problem that the passenger cannot enter when the passenger is far away from the elevator door when the passenger gets on the elevator, which causes the time of getting on and getting off to be long and affects the operation efficiency of the elevator. SUMMARY

[0003] In order to better solve the above problems, the present application provides a visual detection method based on an intelligent camera, which comprises:

[0004] Step S1: acquiring a first image of a passenger entering an elevator and a shooting time of the first image through an internal first camera, and acquiring face information of the passenger based on the first image;

[0005] Step S2: obtaining the predicted floor of getting off of the passenger according to the floor selection image corresponding to the passenger, when the predicted floor of getting off of the passenger cannot be obtained through the floor selection image, searching the historical taking information corresponding to the passenger face information in the storage unit based on the passenger face information and the shooting time of the first image, and obtaining the predicted floor of getting off of the passenger according to the historical taking information of the passenger, the running direction of the elevator and the stop information of the elevator;

[0006] Step S3: repeating the step S1 and the step S2 to obtain the predicted floors of getting off of all passengers in the elevator;

[0007] Step S4: obtaining a second image in the elevator through the first camera, obtaining a first remaining space in the elevator through the second image, obtaining a third image of the floor through the second camera of the elevator, calculating the demand space of the passengers in the floor through the third image, when the demand space is greater than the first remaining space, recalculating a second remaining space by adjusting the standing density of the passengers in the elevator, and determining whether the elevator stops at the floor according to the first remaining space or the second remaining space of the elevator and the demand space of the passengers;

[0008] Step S5: after determining that the elevator stops at the floor and after the passengers get on the elevator, performing the steps S1 to S3 to obtain the predicted floors of getting off of all passengers, obtaining the space occupied by each passenger in the elevator through the second image, dividing the space in the elevator according to the order of getting off, obtaining the optimal getting-off path, and updating the historical taking information of the passengers according to the actual floor of getting off of the passengers and the shooting time of the first image corresponding to the passengers.

[0009] As a preferred technical solution of the present application, in the step S1, the first image of the passenger entering the elevator is obtained through the first camera inside the elevator, and the first image includes the passenger and the object carried by the passenger.

[0010] As a preferred technical solution of the present application, the step S5 further includes a step S6:

[0011] The first camera periodically acquires a second image when the elevator stops at a landing, and the number of passengers in the area of the elevator corresponding to the landing where the elevator stops is obtained based on the second image. The fourth image is also periodically acquired by the first camera, and the running direction and speed of the passengers leaving the elevator can be obtained through the fourth image. The last passenger leaving the elevator is determined based on the number of passengers and the moving direction and speed of the passengers leaving the elevator. The departure time of the last passenger leaving the elevator is calculated according to the moving speed and direction of the last passenger leaving the elevator. The closing time of the elevator door is controlled according to the departure time, and the camera of the first camera can rotate.

[0012] As a preferred technical solution of the present application, the last passenger leaving the elevator is determined based on the number of passengers and the moving direction and speed of the passengers leaving the elevator, comprising the following steps:

[0013] Step S61: Determine the area of the elevator corresponding to the landing where the elevator stops, and guide the passengers to leave the elevator through the ground display unit.

[0014] Step S62: Compare the number of faces in the second image collected in the last cycle with the number of faces in the second image collected this time to obtain the number of face changes. When the number of faces in the second image collected in the last cycle is greater than or equal to 1 and the number of faces in the second image collected this time is 0, the passenger in the fourth image who is moving towards the direction of the elevator door and is closest to the area of the elevator corresponding to the landing is determined as the last passenger leaving the elevator.

[0015] Step S63: Also based on the first pixel position of the fixed feature of the last passenger leaving the elevator in the fourth image collected in the last cycle and the second pixel position of the fixed feature of the last passenger leaving the elevator in the fourth image collected this time, the displacement change is determined based on the first pixel position and the second pixel position. The moving speed of the last passenger leaving the elevator is obtained based on the displacement change and the acquisition period of the fourth image. The departure time is obtained based on the moving speed and the distance of the last passenger leaving the elevator from the elevator door, and the closing time of the elevator door is controlled according to the departure time.

[0016] As a preferred technical scheme of the present application, in the step S4, the initial station density in the elevator is a first density, when the station density is the first density, if the first remaining space is greater than or equal to the demand space, the elevator stops at the landing, if the first remaining space is less than the demand space, the station density is increased to a second density, and the second remaining space is recalculated, if the second remaining space is greater than or equal to the demand space, the elevator stops at the landing, if the second remaining space is less than the demand space, the elevator does not stop at the landing.

[0017] As a preferred technical scheme of the present application, in the step S5, the number of passengers getting off at each landing is obtained based on the predicted landing of all passengers, the space occupied by the passengers and objects carried by the passengers getting off at each landing is obtained based on the third image, the ground of the elevator is regionally divided based on the arrival sequence of each landing and the space occupied by the passengers and objects carried by the passengers getting off at each landing, and the corresponding landing number of each region is displayed to guide passengers to stand at the corresponding position. Meanwhile, the passengers standing at the wrong position are identified through the third image, and the passengers standing at the wrong position are prompted through the display on the ground of the elevator.

[0018] As a preferred technical scheme of the present application, the step S4 further comprises, when the elevator is determined to stop at the landing, the face information of the passenger to be boarded is identified based on the third image, and the historical boarding information corresponding to the face information of the passenger to be boarded is searched in the storage unit based on the shooting time of the third image to determine the predicted landing information of the passenger to be boarded, and the boarding path of the passenger to be boarded is planned by the path planning unit of the elevator based on the predicted landing information.

[0019] As a preferred technical scheme of the present application, in the step S2, the first camera can also rotate an angle, after obtaining the face information of the passenger, the direction of the landing selection panel of the elevator is periodically imaged to obtain a landing selection image, and the landing of the passenger corresponding to the face information of the passenger is determined based on the landing selection panel image of the elevator, wherein the landing selection image obtains the face information of the passenger by extracting the face features.

[0020] The present application also provides a visual detection system based on an intelligent camera, which is used to realize the above-mentioned visual detection method based on a camera, and comprises:

[0021] The first camera unit is used to obtain the first image of the passenger entering the elevator and the shooting time of the first image by the internal first camera, and is also used to obtain the face information of the passenger based on the first image, and is further used to obtain the second image inside the elevator by the first camera, and obtain the first remaining space inside the elevator based on the second image.

[0022] a storage unit configured to store the passenger face information and historical lift information corresponding to the passenger face information;

[0023] The acquisition unit is configured to: acquire the predicted landing floor of the passenger according to the floor selection image corresponding to the passenger, and when the predicted landing floor of the passenger cannot be acquired through the floor selection image, search for the historical lift information corresponding to the passenger face information in the storage unit based on the passenger face information and the shooting time of the first image, and acquire the predicted landing floor of the passenger according to the historical lift information of the passenger, the running direction of the elevator and the stop information of the elevator; and repeat the step S1 and the step S2 to acquire the predicted landing floors of all passengers in the elevator;

[0024] a second camera unit configured to acquire a third image of the floor by a second camera of the elevator, and calculate the demand space of the passengers in the floor by the third image;

[0025] The analysis unit is configured to: when the demand space is greater than the first residual space, calculate a second residual space by adjusting the standing density of the passengers in the elevator again, and determine whether the elevator stops at the floor according to the first residual space or the second residual space of the elevator and the demand space of the passengers;

[0026] The path planning unit is configured to: after the passengers board the elevator, divide the space in the elevator into regions according to the predicted landing floors of all passengers and the space occupied by each passenger in the elevator in the order of landing, and acquire an optimal landing path.

[0027] The ground display unit is configured to display the region division and the optimal landing path, and display passenger standing error prompt information.

[0028] The application further provides a storage medium, which stores program instructions, wherein the program instructions control a device where the storage medium is located to execute the camera-based visual detection method when the program instructions are executed.

[0029] Compared with the prior art, the application has at least the following advantages:

[0030] The application acquires the passenger's face information according to the first image, finds the boarding information corresponding to the passenger's face information in the storage unit and acquires the passenger's expected floor, divides the elevator ground into areas according to the passenger's expected floor so that the passengers can stand in the corresponding area, determines the passenger who leaves the elevator last by the number of passengers in the corresponding area when leaving, acquires the time when the passenger who leaves the elevator last by the fourth image, thereby controlling the elevator door to close quickly after the passenger who leaves the elevator last, shortening the leaving time and improving the efficiency of getting off, calculating the first remaining space in the elevator according to the second image, adjusting the standing density of the passengers in the elevator when the first remaining space is less than the required space of the floor waiting passengers, recalculating the second remaining space, judging whether to stop at the floor by comparing the size relationship between the second remaining space and the required space, maximizing the satisfaction of the passengers, reducing the stopping times, improving the operation efficiency of the elevator, avoiding the problem that the elevator stops at each floor when the load is less than the rated load but the space in the elevator is insufficient, and when it is determined to stop at the floor, acquiring the face information of the passengers waiting at the floor by the third image and finding the historical boarding information corresponding to the face information and the third image shooting time in the storage unit, planning the boarding path and area division of the passengers in advance, shortening the boarding time of the passengers, thereby further improving the operation efficiency of the elevator, and updating the historical boarding information in the storage unit by the actual floor of the passengers, the above technical scheme improves the elevator stopping efficiency by detecting the space in the elevator through the second image, improves the boarding and alighting efficiency by dividing the area through the second image, saves the boarding time through the mutual cooperation of the above scheme, and improves the work efficiency during the entire operation of the elevator. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 A flowchart of a visual detection method based on an intelligent camera according to the application;

[0032] Figure 2 A structural diagram of a visual detection system based on an intelligent camera according to the application;

[0033] Figure 3 A schematic diagram of an elevator according to the application. DETAILED DESCRIPTION

[0034] In order to make the objectives, technical solutions and advantages of the application clearer, the application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and do not limit the application.

[0035] The application provides a visual detection method based on an intelligent camera, as shown in Figure 1 The method comprises:

[0036] Step S1: obtaining a first image of passengers entering the elevator and a shooting time of the first image by an internal first camera, and obtaining passenger face information based on the first image; specifically, the face information includes the contour and geometric features of the face, and the geometric features refer to the geometric relationship between facial features such as eyes, nose, and mouth, such as distance, area, and angle.

[0037] Step S2: selecting an expected floor of the passenger to get off the elevator according to a floor selection image corresponding to the passenger, when the expected floor of the passenger to get off the elevator cannot be obtained through the floor selection image, searching for historical boarding information corresponding to the passenger face information in a storage unit based on the passenger face information and the shooting time of the first image, and obtaining the expected floor of the passenger to get off the elevator according to the historical boarding information of the passenger, the running direction of the elevator, and the stop information of the elevator; specifically, when the passenger enters the elevator to select a floor, the first camera can make the shooting area of the camera align with the floor selection panel of the elevator to shoot a floor selection image through the rotation angle, extract the face features through the floor selection image to obtain the face information, and obtain the floor selected by the corresponding passenger through the floor selection image and the above face information, after the target floor selected by the passenger has been selected, the passenger will not select the floor to get off the elevator again, when there are multiple floors selected by the passenger to get off the elevator, it cannot be determined which floor corresponds to the passenger, because the boarding behavior of the passenger has similarity, therefore, the expected floor of the passenger to get off the elevator can be accurately determined through the historical boarding information of the passenger, the stop information of the elevator obtained through the floor selection image, and the running direction of the elevator obtained from the communication unit of the elevator, wherein the stop information of the elevator is the floor selected by the passenger to get off the elevator on the floor selection panel of the elevator, for example: the time points of going up and down the building are relatively fixed because the time of the passenger going to work or school is relatively fixed.

[0038] Step S3: repeating the step S1 and the step S2 to obtain the expected floor of all passengers in the elevator to get off the elevator;

[0039] Step S4: obtaining a second image in the elevator through the first camera, obtaining a first remaining space in the elevator through the second image, obtaining a third image of the floor through a second camera of the elevator, calculating a demand space of the passengers in the floor through the third image, when the demand space is greater than the first remaining space, recalculating a second remaining space by adjusting the standing density of the passengers in the elevator, and determining whether the elevator stops at the floor according to the first remaining space or the second remaining space of the elevator and the demand space of the passengers.

[0040] Specifically, the second camera comprises a depth camera, the third image comprises depth information, the second camera is arranged outside the elevator, the demand space of the waiting passenger comprises a space occupied by the passenger himself and a space occupied by an object carried by the passenger, a default density of the station position in the elevator is the first density in an initial state, when the first residual space in the elevator calculated through the second image is greater than the demand space of the landing passenger, a greater second residual space is obtained by increasing the density of the station position of the passenger, and whether to stop at the landing is determined according to the size relationship between the second residual space and the demand space, so as to avoid the problem that the passenger cannot be carried once after stopping and needs to be carried multiple times, and the operation efficiency of the elevator is improved.

[0041] Step S5: after determining that the elevator stops at the landing and after the passenger boards the elevator, steps S1 to S3 are executed to obtain the predicted floor of each passenger to get off according to the obtained predicted floor of each passenger to get off, the space occupied by each passenger in the elevator is obtained through the second image, the space in the elevator is divided into regions according to the order of getting off, and the optimal getting-off path is obtained, and the historical boarding information of the passenger is updated according to the actual floor of the passenger to get off and the shooting time of the first image corresponding to the passenger.

[0042] Specifically, as shown in Figure 3 the floor of the elevator is divided into different regions according to the predicted floor of each passenger to get off and the order of getting off at each elevator landing, wherein the region corresponding to the landing that arrives first is closest to the elevator door, and the distance between the region corresponding to the landing and the elevator door becomes farther as the landing arrives later, wherein the landing corresponding to A1 and A2 arrives earlier than the landing corresponding to A3 and A4, the size of the corresponding region is determined according to the number of passengers getting off at each landing, and the above region division is updated after each passenger at the landing gets off, so as to prevent the situation that the passengers need to squeeze from a position far away from the elevator to the elevator door when getting off the elevator, or even the situation that the passengers at the door need to get off first and the passengers behind them can get off, which is particularly inconvenient for old people, children, pregnant women and other special groups, and also causes a longer getting-off time and lower operation efficiency of the elevator. Therefore, according to the stopping order of the elevator, the floor area of the elevator is divided and the path is planned, so that the region corresponding to the landing where the elevator stops is located closest to the elevator door when the elevator stops, so that the passengers can quickly and conveniently get off the elevator, the getting-off time is shortened, and the operation efficiency of the elevator is improved.

[0043] Further, in step S1, the first image of the passenger entering the elevator is obtained through the internal first camera, and the first image comprises the passenger and an object carried by the passenger.

[0044] Further, after step S5, step S6 is further included:

[0045] The first camera periodically acquires a second image when the elevator stops at a landing, and the number of passengers in the elevator corresponding to the area of the landing where the elevator stops is obtained based on the second image. The fourth image is periodically acquired by the first camera, and the moving direction and speed of the passengers leaving the elevator can be obtained through the fourth image. The last passenger leaving the elevator is determined based on the number of passengers and the moving direction and speed of the passengers leaving the elevator. The departure time of the last passenger leaving the elevator is calculated according to the moving speed and direction of the last passenger leaving the elevator. The closing time of the elevator door is controlled according to the departure time. The camera of the first camera can rotate. Specifically, when the elevator stops, the number of faces is obtained by comparing the second image collected in the last period with the second image collected in the current period. When the number of faces in the second image collected in the last period is greater than or equal to 1 and the number of faces collected in the current period is 0, it is confirmed that the passenger corresponding to the face in the second image collected in the last period is the last passenger. The face information collected in the last period is identified. When the number of faces is greater than 1, the distance between the passenger corresponding to the face information and the area where the passenger gets off is determined to determine which passenger is the last passenger. The fourth image is also captured by the first camera aimed at the area near the elevator door. The pixel position change of the inherent characteristics of the last passenger is obtained by comparing the fourth images collected in the adjacent two periods based on the inherent characteristics of the last passenger. The moving direction and speed of the last passenger are calculated by the pixel position change. The time when the last passenger leaves the elevator is calculated by the moving direction and speed of the last passenger. The elevator door is closed quickly after the last passenger leaves the elevator, solving the problem that the existing elevator needs to stay for a period of time after the passenger leaves the elevator door to see if there are still passengers to leave. Time is saved, and the running time of the elevator is further improved.

[0046] Further, determining the last passenger leaving the elevator based on the number of passengers and the moving direction and speed of the passengers leaving the elevator includes the following steps:

[0047] Step S61: Determine the area where the passenger gets off in the elevator corresponding to the landing where the elevator stops, and guide the passenger to get off through the ground display unit. Specifically, when the elevator stops, the area where the passenger gets off is indicated on the ground corresponding to the landing where the elevator stops through the ground display unit, reminding the passenger to prepare to get off and improving the speed of getting off.

[0048] Step S62: face recognition comparison between the second image collected in the last cycle and the second image collected this time to obtain the change in the number of faces; when the number of faces in the second image collected in the last cycle is greater than or equal to 1 and the number of faces in the second image collected this time is 0, the passenger in the fourth image who is moving towards the elevator door direction and is closest to the landing area corresponding to the landing is determined as the passenger who last left the elevator;

[0049] Step S63: further based on the first pixel position of the fixed feature of the passenger who last left the elevator in the fourth image collected in the last cycle and the second pixel position of the fixed feature of the passenger who last left the elevator in the fourth image collected this time, based on the fact that the first pixel position and the second pixel position can indeed change by a displacement amount, based on the displacement amount and the collection cycle of the fourth image, the moving speed of the passenger who last left the elevator is obtained, based on the moving speed and the distance of the passenger who last left the elevator from the elevator door, the leaving time is obtained, and the closing time of the elevator door is controlled according to the leaving time. Specifically, when the existing elevator closes after the passenger gets off, it will stay for a period of time to confirm that the passenger has finished getting off even if there is no passenger getting off, through the above scheme, the time of the passenger who last left the elevator can be determined, the elevator door can be quickly closed when the passenger who last left the elevator leaves the elevator, the waiting time is reduced, and the operation efficiency of the elevator is improved.

[0050] Further, in the step S4, the initial station density in the elevator is the first density, when the station density is the first density, in the case that the first remaining space is greater than or equal to the demand space, the landing is stopped at the landing, in the case that the first remaining space is less than the demand space, the station density is increased to the second density, and the second remaining space is recalculated, in the case that the second remaining space is greater than or equal to the demand space, the landing is stopped at the landing, in the case that the second remaining space is less than the demand space, the landing is not stopped at the landing. Specifically, in the case that the first remaining space is less than the demand space, more remaining space is obtained by adjusting the station density, the passenger's demand for taking the elevator is maximized, and at the same time, the situation that each landing still stops when the load is less than the rated load but there is no space inside the elevator is avoided, the number of elevator stops is reduced, and the operation efficiency of the elevator is improved.

[0051] Further, in the step S5, the number of passengers corresponding to each floor is obtained based on the predicted floor of all passengers, and the space occupied by the passengers and objects carried by the passengers in each floor is obtained according to the third image. The ground of the elevator is regionally divided based on the arrival order of each floor and the space occupied by the passengers and objects carried by the passengers in each floor, and the floor information corresponding to each region is displayed on the ground of the elevator to guide the passengers to stand in the corresponding position. Meanwhile, the passengers standing in the wrong position are identified through the second image, and the passengers are prompted through the ground display of the elevator. Specifically, the number of regions is divided according to the number of floors of the predicted floor, and the size of the region corresponding to each floor is determined according to the space occupied by the passengers and objects carried by the passengers in each floor. The position of the region corresponding to each floor is determined according to the arrival order of each floor. According to the arrival order of the floor, the distance from the elevator door to the region corresponding to the floor is distributed from near to far, that is, the distance from the elevator door to the region corresponding to the floor that arrives first is the shortest, and the distance from the elevator door to the region corresponding to the floor that arrives last is the farthest. The passengers are further guided to stand in the correct position through the second image. The predicted floor of the passenger corresponding to the face information of the passenger and the region corresponding to the floor where the passenger stands are verified. When the floor corresponding to the region where the passenger stands is consistent with the predicted floor of the passenger, the standing position of the passenger is correct, otherwise, the standing position of the passenger is incorrect. The passenger is prompted through the ground display to stand in the appropriate position, which shortens the time of getting off the elevator and improves the speed of getting off the elevator.

[0052] Further, the step S4 further includes, when the elevator determines to stop at the floor, searching for the historical boarding information corresponding to the face information of the passenger to be boarded in the storage unit according to the face information of the passenger to be boarded and the shooting time of the third image, and determining the predicted floor information of the passenger to be boarded. The path planning unit of the elevator plans the boarding path of the passenger to be boarded based on the predicted floor information.

[0053] Specifically, the boarding path is planned according to the target floor of the passenger to be boarded in the floor, that is, the predicted floor. More specifically, the passengers in the region corresponding to the predicted floor in the elevator are boarded first, and the path from the elevator door to the target region is displayed through the ground display unit. The passengers are boarded in the order of the distance from the elevator door to the region corresponding to the predicted floor of the passenger. This avoids the problem that the elevator door is full of passengers and the passengers behind cannot board the elevator, improves the boarding efficiency, and thus improves the operation efficiency of the elevator.

[0054] Further, in the step S2, the first camera can also rotate an angle, and after obtaining the passenger face information, periodically acquire a landing selection image of a landing selection panel of the elevator, determine a landing floor of the passenger corresponding to the passenger face information based on the landing selection panel image of the elevator, and obtain passenger face information by extracting face features from the landing selection image.

[0055] The application further provides a visual detection system based on an intelligent camera, which is used to implement the above-mentioned camera-based visual detection method. Figure 2 As shown in the figure, the system comprises:

[0056] A first camera unit is configured to acquire a first image of a passenger entering an elevator and a shooting time of the first image by a first camera inside the elevator, and obtain passenger face information based on the first image; and acquire a second image inside the elevator by the first camera, and obtain a first remaining space inside the elevator based on the second image.

[0057] A storage unit is configured to store passenger face information and historical boarding information corresponding to the passenger face information.

[0058] The acquisition unit is configured to acquire a predicted landing floor of the passenger according to a landing selection image corresponding to the passenger, and when the predicted landing floor of the passenger cannot be acquired according to the landing selection image, search for historical boarding information corresponding to the passenger face information in the storage unit based on the passenger face information and the shooting time of the first image, and acquire the predicted landing floor of the passenger according to the historical boarding information of the passenger, a running direction of the elevator and stop information of the elevator; and repeat the step S1 and the step S2 to acquire predicted landing floors of all passengers inside the elevator.

[0059] A second camera unit is configured to acquire a third image of a landing by a second camera of the elevator, and calculate a required space of a passenger in the landing based on the third image.

[0060] An analysis unit is configured to, when the required space is greater than the first remaining space, calculate a second remaining space by adjusting a standing density of the passengers in the elevator, and determine whether the elevator stops at the landing according to the first remaining space or the second remaining space of the elevator and the required space of the passenger.

[0061] A path planning unit is configured to, after the passengers board the elevator, divide the space in the elevator into regions according to a descending order of the predicted landing floors of the passengers and the space occupied by each passenger in the elevator, and acquire an optimal descending path.

[0062] The ground display unit is used for displaying the area division and the optimal down-elevator path, and is also used for displaying the passenger station error prompt information.

[0063] The application acquires the passenger's face information according to the first image, and finds the corresponding elevator taking information of the passenger's face information in the storage unit and acquires the passenger's expected down-elevator floor station, divides the elevator ground according to the passenger's expected down-elevator floor station, so that the passengers can stand in the corresponding area, and determines the last passenger leaving the elevator according to the number of passengers in the corresponding area when the passenger is leaving, and acquires the time of the last passenger leaving the elevator through the fourth image, so as to control the elevator door to close quickly after the last passenger leaves, shorten the time of leaving the elevator, improve the efficiency of getting off, and calculate the first remaining space in the elevator according to the second image, and when the first remaining space is less than the required space of the floor station, adjust the station density of the passengers in the elevator, recalculate the second remaining space, and determine whether to stop at the floor station by comparing the size relationship between the second remaining space and the required space, so as to maximize the passenger's demand, reduce the number of stops, improve the operation efficiency of the elevator, avoid the problem that the elevator stops at each floor station when the load is less than the rated load but the space in the elevator is insufficient, and when it is determined to stop at the floor station, acquire the face information of the passengers waiting at the floor station through the third image and find the historical elevator taking information corresponding to the face information and the third image shooting time in the storage, plan the elevator taking path and area division in advance, shorten the time of the passengers taking the elevator, and further improve the operation efficiency of the elevator, and update the historical elevator taking information in the storage unit through the actual down-elevator floor station of the passengers, the above technical scheme improves the elevator stopping efficiency through the second image, and improves the efficiency of taking and leaving the elevator through the area division of the second image, and the above schemes are matched to save the elevator taking time and improve the work efficiency during the entire operation of the elevator.

[0064] The technical features of the above embodiments can be combined in any manner, and to make the description concise, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the description.

[0065] The above embodiments only express several implementation manners of the application, and the description is more specific and detailed, but it should not be understood as a limitation on the patent scope of the application. It should be pointed out that for ordinary skilled persons in the art, some modifications and improvements can be made without departing from the concept of the application, and these belong to the protection scope of the application. Therefore, the protection scope of the patent of the application should be subject to the appended claims.

[0066] The above merely is the preferred embodiment of the present application, and is not used to limit the present application, any modification, equivalent replacement and improvement etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for visual inspection based on smart camera, characterized in that, The method comprises: Step S1: acquiring a first image of passengers entering the elevator and a shooting time of the first image by a first camera inside the elevator, and acquiring passenger face information based on the first image; Step S2: selecting an image of a passenger corresponding to a landing to acquire a predicted landing floor of the passenger, when the predicted landing floor of the passenger cannot be acquired through the image of the landing, searching for historical boarding information corresponding to the passenger face information in a storage unit based on the passenger face information and the shooting time of the first image, and acquiring the predicted landing floor of the passenger according to the historical boarding information of the passenger, a running direction of the elevator and stop information of the elevator; Step S3: repeating the step S1 and the step S2 to acquire predicted landing floors of all passengers inside the elevator; Step S4: acquiring a second image inside the elevator by the first camera, acquiring a first remaining space inside the elevator through the second image, acquiring a third image of a landing by a second camera of the elevator, calculating a demand space of passengers at the landing through the third image, when the demand space is greater than the first remaining space, recalculating a second remaining space by adjusting a standing density of passengers in the elevator, and determining whether the elevator stops at the landing according to the first remaining space or the second remaining space of the elevator and the demand space of the passengers; Step S5: after it is determined that the elevator stops at the landing and after the passengers board the elevator, performing the steps S1 to S3 to acquire predicted landing floors of all passengers, acquiring a space occupied by each passenger in the elevator through the second image, and dividing the space in the elevator into regions according to a descending order and acquiring an optimal descending path, and updating historical boarding information of the passengers according to actual landing floors of the passengers and the shooting time of the first image corresponding to the passengers. 2.The visual inspection method based on the intelligent camera according to claim 1, wherein, In the step S1, the first image of the passengers entering the elevator is acquired by the first camera inside the elevator, and the first image includes the passengers and objects carried by the passengers. 3.The visual inspection method based on the intelligent camera of claim 2, wherein, The step S5 further comprises a step S6: When the elevator stops at the landing, the first camera periodically acquires a second image, the number of passengers in a region corresponding to the landing where the elevator stops is acquired based on the second image, the first camera periodically acquires a fourth image, the running direction and speed of passengers leaving the elevator are acquired through the fourth image, the last passenger leaving the elevator is determined based on the number of passengers and the moving direction and speed of the passengers leaving the elevator, the leaving time of the last passenger leaving the elevator is calculated according to the moving speed and direction of the last passenger leaving the elevator, and the closing time of the elevator door is controlled according to the leaving time, wherein the camera of the first camera can rotate. The last passenger leaving the elevator is determined based on the number of passengers and the moving direction and speed of the passengers leaving the elevator, and comprises the following steps:

4. The method of claim 3, wherein the method further comprises: Step S61: determining a descending region in the elevator corresponding to the landing where the elevator stops through the landing, and guiding the passengers to descend through a ground display unit. ​ Step S62: the second image collected in the last cycle is compared with the second image collected this time to obtain the number of faces, and when the number of faces in the second image collected in the last cycle is greater than or equal to 1 and the number of faces in the second image collected this time is 0, the passenger in the fourth image who is moving towards the elevator door direction and is closest to the landing area corresponding to the landing is determined as the passenger who last left the elevator; Step S63: the first pixel position of the fixed feature of the passenger who last left the elevator in the fourth image collected in the last cycle and the second pixel position of the fixed feature of the passenger who last left the elevator in the fourth image collected this time are also used to determine whether the first pixel position and the second pixel position can be displaced by a displacement change amount, to obtain the moving speed of the passenger who last left the elevator based on the displacement change amount and the collection cycle of the fourth image, to obtain the leaving time based on the moving speed and the distance of the passenger who last left the elevator from the elevator door, and to control the closing time of the elevator door according to the leaving time.

5. The method of claim 1, wherein the method further comprises: In the step S4, the initial station density in the elevator is a first density, and when the station density is the first density, the landing is stopped at the landing when the first remaining space is greater than or equal to the demand space, and the station density is increased to a second density when the first remaining space is less than the demand space, and the second remaining space is recalculated, and the landing is stopped at the landing when the second remaining space is greater than or equal to the demand space, and the landing is not stopped at the landing when the second remaining space is less than the demand space, wherein the second density is greater than the first density.

6. The method of claim 1, wherein the method further comprises: In the step S5, the number of passengers who get off at each landing is obtained based on the predicted landing floors of all passengers, and the space occupied by the passengers and objects carried by the passengers who get off at each landing is also obtained based on the second image, the floor of the elevator is regionally divided based on the arrival sequence of each landing and the space occupied by the passengers and objects carried by the passengers who get off at each landing, and the corresponding landing floor number of each region is displayed to guide passengers to stand at the corresponding position, and passengers who stand at the wrong position are also identified through the second image, and prompt information is displayed on the floor of the elevator when passengers stand at the wrong position.

7. The method of claim 1, wherein the method further comprises: The step S4 further comprises, when the elevator determines to stop at the landing, searching for the historical boarding information corresponding to the face information of the passenger who is about to board the elevator in the storage unit based on the face information of the passenger who is about to board the elevator identified by the third image and the shooting time of the third image, and the boarding path of the passenger who is about to board the elevator is planned by the path planning unit of the elevator based on the predicted landing floor information of the passenger who is about to board the elevator. 8.The method of claim 1, wherein, In the step S2, after obtaining the passenger face information, the image of the landing selection panel of the elevator is periodically collected to obtain a landing selection image, and the landing floor of the passenger corresponding to the passenger face information is determined based on the landing selection panel image of the elevator, wherein the passenger face information is obtained by extracting face features, and the first camera can rotate.

9. A smart camera-based vision inspection system for implementing the camera-based vision inspection method of any one of claims 1-8, characterized in that, The system comprises: A first camera unit is configured to acquire a first image of a passenger entering an elevator and a shooting time of the first image by a first camera inside the elevator, and to acquire face information of the passenger based on the first image, and to acquire a second image inside the elevator by the first camera, and to acquire a first remaining space inside the elevator by the second image; A storage unit is configured to store the face information of the passenger and historical elevator-boarding information corresponding to the face information of the passenger; An acquisition unit is configured to acquire an estimated floor of alighting of the passenger according to a floor selection image of the passenger, and to acquire historical elevator-boarding information corresponding to the face information of the passenger in the storage unit based on the face information of the passenger and the shooting time of the first image when the estimated floor of alighting of the passenger cannot be acquired through the floor selection image, and to acquire the estimated floor of alighting of the passenger according to the historical elevator-boarding information of the passenger, a running direction of the elevator and stop information of the elevator, and to acquire the estimated floor of alighting of all passengers in the elevator by repeating step S1 and step S2; A second camera unit is configured to acquire a third image of a floor by a second camera of the elevator, and to calculate a required space of passengers at the floor by the third image; An analysis unit is configured to calculate a second remaining space by adjusting a standing density of the passengers in the elevator when the required space is greater than the first remaining space, and to determine whether the elevator stops at the floor according to the first remaining space or the second remaining space of the elevator and the required space of the passengers; A path planning unit is configured to divide the space in the elevator into regions and acquire an optimal alighting path according to the estimated floor of alighting of all passengers and the space occupied by each passenger in the elevator in the order of alighting after the passengers board the elevator; A ground display unit is configured to display the region division and the optimal alighting path, and to display a standing error prompt information of the passengers.

10. A storage medium, characterized by The storage medium stores program instructions, wherein the program instructions control a device where the storage medium is located to execute the camera-based visual inspection method in any one of claims 1 to 8 when the program instructions are executed.

Images