Method, device and equipment for judging state of personnel in elevator based on elevator camera and medium

Abstract

The invention belongs to the technical field of intelligent monitoring and elevator safety, and particularly provides a method, device and equipment for judging the state of personnel in an elevator based on elevator cameras and a medium, and the method comprises the following steps: collecting picture data collected by different elevator cameras to generate a data set; the data set comprises image data of a falling behavior and a non-falling behavior; frame-pulling labeling is carried out on different states in the picture, and a labeling file containing a target bounding box and a category label is generated; re-clustering is carried out according to an annotation box of the data set to obtain a new Anchor, and YOLOv5 training is adopted to obtain a falling-to-ground identification model; capturing a video stream through a camera arranged on the elevator and preprocessing the video stream; and the processed data are input into a falling identification model, when the falling state of the elevator taking personnel is identified and displayed, early warning information is sent to the background service personnel of the elevator, and meanwhile, the real-time picture of the elevator is switched to the background service personnel. And rescue measures can be quickly started, and the response time is shortened.

Description

Background Art

[0002] As a vertical transportation tool connecting various floors in modern buildings, elevators have not only greatly improved people's travel efficiency, but also promoted the rational use and development of urban space. However, with the surge in the frequency of elevator use, various accidents inside elevators have gradually increased, especially at night or during periods with less traffic. If passengers suddenly faint due to illness or accidentally fall, and other emergencies are not discovered and handled in time, they may pose a serious threat to the safety of passengers' lives.

[0003] Traditional elevator monitoring systems, although generally equipped with video monitoring equipment that can record images inside and outside the elevator, are mostly limited to simple recording and playback functions, lacking the ability to analyze and judge the status of people inside the elevator in real time. This means that even if an abnormal situation occurs in the monitoring image, it is necessary to rely on manual viewing of the video to identify it, which is not only inefficient, but also often has a lag, making it difficult to detect and respond to emergency situations in the elevator in the first place. Summary of the invention

[0004] In view of the problem that abnormal situations appear in the elevator monitoring screen and it is difficult to discover and respond to the emergency situation in the elevator in the first time, the present invention provides a method, device, equipment and medium for judging the status of people in the elevator based on the elevator camera.

[0005] In a first aspect, the technical solution of the present invention provides a method for determining the status of people inside an elevator based on an elevator camera, comprising the following steps: The image data of people of different ages, genders, and states in the elevator collected by different elevator cameras are collected to generate a data set; the data set includes image data of falling behavior and non-falling behavior; Draw boxes to mark different states in the image and generate a labeling file containing the target bounding box and category label; Re-clustering is performed based on the labeled boxes of the dataset to obtain new Anchors, and YOLOv5 is used to train the fall recognition model. Capture video streams through elevator cameras and pre-process the video streams; The processed image frames are input into the fall recognition model. When the fall status of the elevator occupant is identified, an early warning message is sent to the elevator backstage service personnel, and the real-time elevator image is cut to the backstage service personnel.

[0006] After receiving the fall recognition confirmation information input by the backend service personnel, the image data input to the model is automatically annotated into the VOC data format, and the data set is incrementally updated.

[0007] As a further limitation of the technical solution of the present invention, the steps of re-clustering the annotation boxes of the data set to obtain new Anchors and using YOLOv5 training to obtain a fall recognition model include: Preprocess the data set and divide the processed data set into a training set and a validation set; Re-cluster according to the annotation box of the data set to obtain a new Anchor; Replace the default Anchor in the YOLOv5 model file with the re-clustered Anchor; Define the loss function; Initialize the YOLOv5 model and set training parameters; Use the prepared training set and the defined loss function to train the model. The training process uses a convolutional neural network to extract image features and uses the back propagation algorithm to update the model weights. Use the validation set to evaluate the performance of the model, and adjust and optimize the model based on the evaluation results to obtain a trained fall recognition model.

[0008] As a further limitation of the technical solution of the present invention, the step of preprocessing the data set includes: Convert the image data into VOC data format, normalize the image, and convert the pixel value to between 0 and 1; Perform data augmentation operations.

[0009] As a further limitation of the technical solution of the present invention, the step of re-clustering the annotation boxes of the preprocessed data set to obtain new Anchors includes: Extract annotation box information: Extract bounding box size information of all objects from the annotation file; Select the number of clusters K and perform K-means clustering on the bounding box size. After clustering is completed, the center point of each cluster will become a new anchor.

[0010] As a further limitation of the technical solution of the present invention, the convolution feature extraction formula is as follows:

[0011] Where padding is the padding size, which indicates the number of layers of zeros added around the borders of the input image, kernel_size is the size of the convolution kernel, stride is the stride of the convolution operation, which indicates the number of pixels that the convolution kernel moves on the input image; input_size is the size of the input image or feature map;

[0012] where output_size is the desired output size.

[0013] As a further limitation of the technical solution of the present invention, the method further includes: The model training process uses the cross entropy loss function to calculate the classification error of the target category and the regression error of the bounding box; Cross entropy loss function L = - [y * log(p) + (1 - y) * log(1 - p)] Where: L represents the loss value, y represents the true label of the sample, which takes a value of 0 or 1, and p represents the probability that the model predicts it to be a positive class.

[0014] As a further limitation of the technical solution of the present invention, the steps of capturing a video stream by an elevator camera and processing the video stream include: Capture video streams through cameras set up in elevators; Decode the video stream and convert it into a raw image frame sequence; The decoded image frames are converted into the VOC data format that the model can recognize; Scale the image frame according to the model's input size requirements; Use a denoising algorithm to denoise the image and perform edge detection on the image; A frame selection algorithm is used to filter out image frames containing elevator personnel.

[0015] In a second aspect, the technical solution of the present invention also provides a device for judging the status of people inside an elevator based on an elevator camera, comprising a data collection module, a labeling file generation module, a model training module, a video processing module, an early warning processing module and a data update module; The data collection module is used to collect image data of people of different ages, genders, and states in the elevator collected by different elevator cameras to generate a data set; the data set includes image data of falling behavior and non-falling behavior; The annotation file generation module is used to draw boxes to annotate different states in the image and generate annotation files containing target bounding boxes and category labels; The model training module is used to re-cluster the annotation boxes of the data set to obtain new anchors, and use YOLOv5 training to obtain the fall recognition model; A video processing module, used for capturing a video stream through a camera installed in the elevator and pre-processing the video stream; The early warning processing module is used to input the processed data into the fall recognition model, and when the fall status of the elevator passenger is identified, an early warning message is sent to the elevator backstage service personnel, and the real-time image of the elevator is cut to the backstage service personnel; The data update module is used to automatically mark the model input data as VOC data format after receiving the fall recognition confirmation information input by the backend service personnel, and to incrementally update the data set.

[0016] As a further limitation of the technical solution of the present invention, the model training module includes a preprocessing unit, a clustering processing unit, a training unit and a verification unit; A preprocessing unit, used for preprocessing the data set and dividing the processed data set into a training set and a validation set; The clustering processing unit is used to re-cluster the annotation boxes of the data set to obtain new Anchors; the re-clustered Anchors replace the default Anchors in the YOLOv5 model file; The training unit is used to define the loss function; initialize the YOLOv5 model and set the training parameters; use the prepared training set and the defined loss function to train the model. The training process uses a convolutional neural network to extract image features and uses the back propagation algorithm to update the model weights; The verification unit is used to evaluate the performance of the model using the verification set, and adjust and optimize the model according to the evaluation results to obtain a trained fall recognition model.

[0017] As a further limitation of the technical solution of the present invention, the preprocessing unit is specifically used to convert the image data into the VOC data format, normalize the image, convert the pixel value to between 0 and 1, and perform data enhancement operations.

[0018] As a further limitation of the technical solution of the present invention, the clustering processing unit is specifically used to extract the annotation box information: extract the bounding box size information of all targets from the annotation file; select the clustering cluster number K value, perform K-means clustering on the bounding box size, and after the clustering is completed, the center point of each cluster will become a new Anchor.

[0019] As a further limitation of the technical solution of the present invention, the convolution feature extraction formula is as follows:

[0020] Where padding is the padding size, which indicates the number of layers of zeros added around the borders of the input image, kernel_size is the size of the convolution kernel, stride is the stride of the convolution operation, which indicates the number of pixels that the convolution kernel moves on the input image; input_size is the size of the input image or feature map;

[0021] where output_size is the desired output size.

[0022] As a further limitation of the technical solution of the present invention, the model training process uses a cross entropy loss function to calculate the classification error of the target category and the regression error of the bounding box; Cross entropy loss function L = - [y * log(p) + (1 - y) * log(1 - p)] Where: L represents the loss value, y represents the true label of the sample, which takes a value of 0 or 1, and p represents the probability that the model predicts it to be a positive class.

[0023] The video processing module is specifically used to capture the video stream through the camera set up in the elevator; decode the video stream and convert it into the original image frame sequence; convert the decoded image frame into the VOC data format that the model can recognize; scale the image frame according to the input size requirements of the model; use the denoising algorithm to denoise the image and perform edge detection on the image; use the frame selection algorithm to filter out the image frames containing elevator personnel.

[0024] In a third aspect, the technical solution of the present invention also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores computer program instructions executable by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the method for determining the status of people inside an elevator based on an elevator camera as described in the first aspect.

[0025] In a fourth aspect, the technical solution of the present invention also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions enable the computer to execute the method for determining the status of people inside an elevator based on an elevator camera as described in the first aspect.

[0026] It can be seen from the above technical solutions that the present invention has the following advantages: by real-time monitoring of the internal situation of the elevator, especially being able to quickly identify whether the passengers have fallen, the safety of elevator use can be greatly improved. Once the falling state is identified, an early warning message is immediately sent to the elevator back-end service personnel, which helps to quickly start rescue measures, shorten the response time, and reduce potential risks. The use of advanced deep learning models such as YOLOv5 for training makes the falling identification process more intelligent and automated. By re-clustering to obtain new Anchors, the recognition accuracy and generalization ability of the model are improved, and the falling behavior of different age groups, different genders and different states can be more accurately identified. By collecting rich image data collected by different elevator cameras, a data set containing multiple states is generated, which provides a solid foundation for the training and optimization of the model. After receiving the recognition confirmation information from the back-end service personnel, the identified falling state data is automatically marked as VOC data format, and the data set is incrementally updated, realizing continuous optimization of data and iterative upgrade of the model.

[0027] The real-time monitoring and early warning mechanism not only improves the safety of elevator use, but also helps improve the service quality of elevator maintenance and emergency response by providing key information to back-end service personnel in a timely manner. Transmitting the real-time elevator image to back-end service personnel helps service personnel understand the on-site situation more intuitively and make more accurate judgments and decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention.

[0030] Figure 2 is a schematic block diagram of an apparatus according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] like Figure 1As shown, an embodiment of the present invention provides a method for determining the status of a person inside an elevator based on an elevator camera, comprising the following steps: Step 1: Collect image data of people of different ages, genders, and states in the elevator collected by different elevator cameras to generate a data set; the data set includes image data of falling behavior and non-falling behavior; Step 2: Draw boxes to mark different states in the image and generate a labeling file containing the target bounding box and category label; Use annotation tools (such as Labelme, etc.) to annotate the images in the dataset and generate annotation files. The annotation files should contain the bounding box coordinates and category labels of each target (human body).

[0033] Step 3: Re-cluster the labeled boxes of the data set to obtain new Anchors, and use YOLOv5 training to obtain the fall recognition model; Step 4: Capture the video stream through the camera set up in the elevator and pre-process the video stream; A high-definition camera is installed in the elevator to ensure that the situation inside the elevator can be clearly captured. The camera is connected to the elevator control system or monitoring center via wired or wireless means to transmit video streams in real time. The captured video stream is decoded and converted into a format that can be processed by the computer. The video frames are scaled, cropped, denoised, etc. to improve image quality and recognition accuracy. The frame rate is adjusted to ensure the smoothness and real-time nature of the video stream.

[0034] Step 5: Input the processed data into the fall recognition model. When the fall status of the elevator passenger is identified, an early warning message is sent to the elevator backstage service personnel, and the real-time image of the elevator is cut to the backstage service personnel.

[0035] In the embodiment of the present invention, the trained model is exported and integrated into the target detection system to realize the real-time detection of falling behavior. Here, the target detection system is included in the monitoring center or the elevator control system, that is, the trained falling recognition model is loaded in the monitoring center or the elevator control system. Ensure that the model is compatible with the currently used hardware and software environment.

[0036] The preprocessed video stream data is input into the fall recognition model frame by frame. The model extracts and classifies features of each frame to determine whether there is a fall behavior. The model outputs the recognition result, including whether there is a fall behavior and the confidence of the fall behavior. When the fall behavior is recognized and the confidence exceeds the preset threshold, the early warning mechanism is triggered. In an embodiment of the present invention, the monitoring center or the elevator control system sends early warning information to the elevator back-end service personnel through communication methods such as text messages, emails, and telephone calls. The early warning information should include key information such as the elevator number, floor, and location of the person who fell. At the same time as the early warning mechanism is triggered, the real-time image of the elevator is switched to the monitoring screen of the elevator back-end service personnel. Ensure that the back-end service personnel can intuitively see the internal situation of the elevator so that they can respond quickly.

[0037] The monitoring center or elevator control system should record the time, location, and recognition results of each fall recognition event. This information can be used for subsequent analysis, statistics, and reporting. The fall recognition model should be continuously optimized and adjusted based on the feedback from back-end service personnel and actual conditions. More diverse data sets should be collected to improve the generalization ability and recognition accuracy of the model.

[0038] Step 6: After receiving the recognition confirmation information from the backend service personnel, the model input data that has been identified as a fallen state by the fall recognition model is automatically annotated into the VOC data format, and the data set is incrementally updated.

[0039] In some embodiments, the steps of re-clustering the labeled boxes of the data set to obtain new Anchors and using YOLOv5 training to obtain a fall recognition model include: Step 31: preprocess the data set and divide the processed data set into a training set and a validation set; In an embodiment of the present invention, a data set containing falling behaviors is collected to ensure that there are enough positive samples (i.e., falling behaviors) and negative samples (i.e., non-falling behaviors) in the data set; the preprocessing process includes: converting the image data into a VOC data format, normalizing the image, and converting the pixel value to between 0 and 1; and performing data enhancement operations.

[0040] Step 32: Re-cluster according to the annotation box of the data set to obtain a new Anchor; Specifically include: Extracting annotation box information: Extract bounding box size information of all objects from the annotation file; Select the K value for the number of clusters and perform K-means clustering on the bounding box size. After clustering is completed, the center point of each cluster will become a new anchor. Evaluate the quality and quantity of anchors based on the clustering results and actual needs. If the clustering effect is not good or the number of anchors is too many / too few, you can adjust the K value and re-cluster. Among them, according to the diversity of targets in the data set, select a suitable K value to represent the number of clusters. The choice of K value will affect the number and distribution of anchors.

[0041] Step 33: Replace the default Anchor in the YOLOv5 model file with the re-clustered Anchor; Here you need to ensure that the system has installed the Python package of YOLOv5 and related dependencies. According to the characteristics and requirements of the dataset, configure the relevant files of YOLOv5, including data path, number of categories, category name, etc. If you need to re-cluster the Anchor, replace the default Anchor in the YOLOv5 model file with the clustered Anchor.

[0042] Step 34: Define the loss function; Here, the model training process uses the cross entropy loss function to calculate the classification error of the target category and the regression error of the bounding box; Cross entropy loss function L = - [y * log(p) + (1 - y) * log(1 - p)] Where: L represents the loss value, y represents the true label of the sample, which takes a value of 0 or 1, and p represents the probability that the model predicts it to be a positive class.

[0043] Step 35: Initialize the YOLOv5 model and set the training parameters; according to the size of the dataset and hardware resources, set appropriate training parameters such as batch size, learning rate, number of training rounds, etc.

[0044] Step 36: Use the prepared training set and the defined loss function to train the model. The training process uses a convolutional neural network to extract image features and uses a back propagation algorithm to update the model weights. Use tools such as TensorBoard to monitor the training process, including changes in loss functions, accuracy, and other indicators. Step 37: Use the validation set to evaluate the performance of the model, and adjust and optimize the model based on the evaluation results to obtain a trained fall recognition model.

[0045] In some embodiments, the convolution feature extraction formula is as follows:

[0046] Where padding is the padding size, which indicates the number of layers of zeros added around the borders of the input image, kernel_size is the size of the convolution kernel, stride is the stride of the convolution operation, which indicates the number of pixels that the convolution kernel moves on the input image; input_size is the size of the input image or feature map;

[0047] where output_size is the desired output size.

[0048] In some embodiments, the steps of capturing a video stream through an elevator camera and processing the video stream include: Step 41: Capture a video stream through a camera set up in the elevator; Step 42: Decode the video stream and convert it into an original image frame sequence; Step 43: The decoded image frame is converted into a VOC data format that can be recognized by the model; Step 44: scaling the image frame according to the input size requirement of the model; Step 45: Use a denoising algorithm (such as median filtering, Gaussian filtering, etc.) to denoise the image and perform edge detection on the image; for example, use a Canny edge detector to perform edge detection; Step 46: Use a frame selection algorithm (such as a frame selection algorithm based on motion detection) to filter out image frames containing elevator personnel.

[0049] like Figure 2 As shown, an embodiment of the present invention also provides a device for judging the status of people inside an elevator based on an elevator camera, including a data collection module, a labeling file generation module, a model training module, a video processing module and an early warning processing module; The data collection module is used to collect image data of people of different ages, genders, and states in the elevator collected by different elevator cameras to generate a data set; the data set includes image data of falling behavior and non-falling behavior; The annotation file generation module is used to draw boxes to annotate different states in the image and generate annotation files containing target bounding boxes and category labels; The model training module is used to re-cluster the annotation boxes of the data set to obtain new anchors, and use YOLOv5 training to obtain the fall recognition model; A video processing module, used for capturing a video stream through a camera installed in the elevator and pre-processing the video stream; The early warning processing module is used to input the processed data into the fall recognition model. When the fall status of the elevator passenger is identified, an early warning message is sent to the elevator backstage service personnel, and the real-time image of the elevator is cut to the backstage service personnel.

[0050] In some embodiments, the device also includes a data update module, which is used to automatically mark the model input data of the fall state identified by the fall recognition model as VOC data format after receiving the recognition confirmation information from the background service personnel, and incrementally update the data set.

[0051] In some embodiments, the model training module includes a preprocessing unit, a clustering processing unit, a training unit, and a verification unit; A preprocessing unit, used for preprocessing the data set and dividing the processed data set into a training set and a validation set; The clustering processing unit is used to re-cluster the annotation boxes of the data set to obtain new Anchors; the re-clustered Anchors replace the default Anchors in the YOLOv5 model file; The training unit is used to define the loss function; initialize the YOLOv5 model and set the training parameters; use the prepared training set and the defined loss function to train the model. The training process uses a convolutional neural network to extract image features and uses the back propagation algorithm to update the model weights; The verification unit is used to evaluate the performance of the model using the verification set, and adjust and optimize the model according to the evaluation results to obtain a trained fall recognition model.

[0052] In some embodiments, the preprocessing unit is specifically used to convert the image data into a VOC data format, normalize the image, convert the pixel value to between 0 and 1, and perform data enhancement operations.

[0053] The clustering processing unit is specifically used to extract the annotation box information: extract the bounding box size information of all targets from the annotation file; select the cluster number K value, and perform K-means clustering on the bounding box size. After the clustering is completed, the center point of each cluster will become a new Anchor.

[0054] In some embodiments, the convolution feature extraction formula is as follows:

[0055] Where padding is the padding size, which indicates the number of layers of zeros added around the borders of the input image, kernel_size is the size of the convolution kernel, stride is the stride of the convolution operation, which indicates the number of pixels that the convolution kernel moves on the input image; input_size is the size of the input image or feature map;

[0056] where output_size is the desired output size.

[0057] The model training process uses the cross entropy loss function to calculate the classification error of the target category and the regression error of the bounding box; Cross entropy loss function L = - [y * log(p) + (1 - y) * log(1 - p)] Where: L represents the loss value, y represents the true label of the sample, which takes a value of 0 or 1, and p represents the probability that the model predicts it to be a positive class.

[0058] The method provided by the present invention deeply analyzes the video data in the elevator to achieve real-time monitoring and accurate judgment of passenger behavior. For example, it can automatically identify abnormal postures of passengers, such as sudden collapse, long-term immobility, and other signs that may indicate a health crisis, and immediately trigger an alarm mechanism to notify management personnel or emergency rescue services, thereby greatly shortening the response time and improving rescue efficiency.

[0059] The embodiment of the present invention also provides an electronic device, which includes: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus. The communication bus can be used for information transmission between the electronic device and the sensor. The processor can call the logic instructions in the memory to execute the following method: collect the image data of different age groups, different genders and different states of people in the elevator collected by different elevator cameras to generate a data set; the data set includes image data of falling behavior and non-falling behavior; draw a box to mark the different states in the picture, and generate a marking file containing a target boundary box and a category label; re-cluster the marked box of the data set to obtain a new Anchor, and use YOLOv5 training to obtain a falling recognition model; capture the video stream through the camera set by the elevator and pre-process the video stream; input the processed data into the falling recognition model, and when the falling state of the elevator passengers is identified, an early warning message is sent to the elevator backstage service personnel, and the real-time picture of the elevator is cut to the backstage service personnel.

[0060] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0061] An embodiment of the present invention provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions enable a computer to execute the method provided by the above method embodiment, for example, including: collecting image data of people of different ages, genders, and states in the elevator collected by different elevator cameras to generate a data set; the data set includes image data of falling behavior and non-falling behavior; marking different states in the image by drawing boxes to generate a marking file containing a target boundary box and a category label; re-clustering the marked boxes of the data set to obtain a new Anchor, and using YOLOv5 training to obtain a falling recognition model; capturing a video stream through a camera set in the elevator and pre-processing the video stream; inputting the processed data into the falling recognition model, and when the falling state of the elevator passenger is identified, a warning message is sent to the elevator backstage service personnel, and the real-time image of the elevator is cut to the backstage service personnel.

[0062] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0063] An embodiment of the device for determining the status of people inside an elevator based on an elevator camera provided in an embodiment of the present invention belongs to the same inventive concept as the method for determining the status of people inside an elevator based on an elevator camera in the above-mentioned embodiments. For details not fully described in the embodiment of the device for determining the status of people inside an elevator based on an elevator camera, reference can be made to the embodiment of the method for determining the status of people inside an elevator based on an elevator camera.

[0064] The device for judging the status of people inside the elevator based on the elevator camera is a unit and algorithm step of each example described in combination with the embodiments disclosed in this article, which can be implemented by electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0065] Those skilled in the art will appreciate that various aspects of the method for determining the status of people inside an elevator based on an elevator camera can be implemented as a system, method or program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which can be collectively referred to as "circuit", "module" or "system" here.

[0066] Although the present invention has been described in detail with reference to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention.

Claims

1. A method for judging the status of people inside an elevator based on an elevator camera, characterized in that: The steps include: The image data of people of different ages, genders, and states in the elevator collected by different elevator cameras are collected to generate a data set; the data set includes image data of falling behavior and non-falling behavior; Draw boxes to mark different states in the image and generate a labeling file containing the target bounding box and category label; Re-clustering is performed based on the labeled boxes of the dataset to obtain new Anchors, and YOLOv5 is used to train the fall recognition model. Capture video streams through elevator cameras and process the video streams; The processed image frames are input into the fall recognition model. When the fall status of the elevator passengers is identified, an early warning message is sent to the elevator backstage service personnel, and the real-time image of the elevator is cut to the backstage service personnel; After receiving the fall recognition confirmation information input by the backend service personnel, the image data input to the model is automatically annotated into the VOC data format, and the data set is incrementally updated.

2. The method for judging the status of people inside an elevator based on an elevator camera according to claim 1, characterized in that: Re-clustering is performed based on the labeled boxes of the data set to obtain new Anchors. The steps of using YOLOv5 to train the fall recognition model include: Preprocess the data set and divide the processed data set into a training set and a validation set; Re-cluster according to the annotation box of the data set to obtain a new Anchor; Replace the default Anchor in the YOLOv5 model file with the re-clustered Anchor; Define the loss function; Initialize the YOLOv5 model and set training parameters; Use the prepared training set and the defined loss function to train the model. The training process uses a convolutional neural network to extract image features and uses the back propagation algorithm to update the model weights. Use the validation set to evaluate the performance of the model, and adjust and optimize the model based on the evaluation results to obtain a trained fall recognition model.

3. The method for judging the status of people inside an elevator based on an elevator camera according to claim 2, characterized in that: The steps to preprocess the dataset include: Convert the image data into VOC data format, normalize the image, and convert the pixel value to between 0 and 1; Perform data augmentation operations.

4. The method for judging the status of people inside an elevator based on an elevator camera according to claim 3, characterized in that: The steps of re-clustering the annotation boxes of the preprocessed data set to obtain new Anchors include: Extract annotation box information: Extract bounding box size information of all objects from the annotation file; Select the number of clusters K and perform K-means clustering on the bounding box size. After clustering is completed, the center point of each cluster will become a new anchor.

5. The method for judging the status of people inside an elevator based on an elevator camera according to claim 4, characterized in that: The method further includes: The model training process uses the cross entropy loss function to calculate the classification error of the target category and the regression error of the bounding box; Cross entropy loss function L = - [y * log(p) + (1 - y) * log(1 - p)] Where: L represents the loss value, y represents the true label of the sample, which takes a value of 0 or 1, and p represents the probability that the model predicts it to be a positive class.

6. The method for judging the status of people inside an elevator based on an elevator camera according to claim 5, characterized in that: The steps of capturing and processing video streams through elevator cameras include: Capture video streams through cameras set up in elevators; Decode the video stream and convert it into a raw image frame sequence; The decoded image frames are converted into the VOC data format that the model can recognize; Scale the image frame according to the model's input size requirements; Use a denoising algorithm to denoise the image and perform edge detection on the image; A frame selection algorithm is used to filter out image frames containing elevator personnel.

7. A device for judging the status of people inside an elevator based on an elevator camera, characterized in that: It includes data collection module, annotation file generation module, model training module, video processing module, warning processing module and data update module; The data collection module is used to collect image data of people of different ages, genders, and states in elevators collected by different elevator cameras to generate a data set; The dataset includes image data of falling behavior and non-falling behavior; The annotation file generation module is used to draw boxes to annotate different states in the image and generate annotation files containing target bounding boxes and category labels; The model training module is used to re-cluster the annotation boxes of the data set to obtain new anchors, and use YOLOv5 training to obtain the fall recognition model; A video processing module, used for capturing a video stream through a camera installed in the elevator and pre-processing the video stream; The early warning processing module is used to input the processed data into the fall recognition model, and when the fall status of the elevator passenger is identified, an early warning message is sent to the elevator backstage service personnel, and the real-time image of the elevator is cut to the backstage service personnel; The data update module is used to automatically mark the model input data as VOC data format after receiving the fall recognition confirmation information input by the backend service personnel, and to incrementally update the data set.

8. The device for judging the status of people inside an elevator based on an elevator camera according to claim 7, characterized in that: The model training module performs the training process using the cross entropy loss function to calculate the classification error of the target category and the regression error of the bounding box; Cross entropy loss function L = - [y * log(p) + (1 - y) * log(1 - p)] Where: L represents the loss value, y represents the true label of the sample, which takes a value of 0 or 1, and p represents the probability that the model predicts it to be a positive class.

9. An electronic device, characterized in that: The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the method for determining the status of people inside an elevator based on an elevator camera as described in any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the method for determining the status of people inside an elevator based on an elevator camera as described in any one of claims 1 to 6.

Images