End-side cloud cooperative elevator car safety monitoring method and related equipment

Abstract

The embodiment of the invention provides an end-side cloud collaborative elevator car safety monitoring method and related equipment, and belongs to the technical field of elevator safety. The method comprises the following steps: acquiring a visual image in an elevator car; image quality judgment is carried out on the visual images, and the visual images are divided into normal illumination images and low illumination images; performing weak light enhancement processing on the low-illumination image to obtain a weak light enhanced image; performing equipment security classification prediction according to the weak light enhanced image to obtain a cloud computing result; passenger safety monitoring is carried out on the normal illumination image through a single-stage target detection algorithm, and an edge calculation result is obtained; and performing safety early warning monitoring according to the edge calculation result and the cloud calculation result to obtain an early warning result. According to the embodiment of the invention, the safety monitoring efficiency and the adaptability to complex environments can be improved, all-weather monitoring and quick response to elevator equipment and passenger safety are realized, and early warning is made in time.

Description

Technical Field

[0001] This application relates to the field of elevator safety technology, and in particular to an edge-cloud collaborative method and related equipment for elevator car safety monitoring. Background Technology

[0002] The development of smart cities is a major issue in modern society. Communities and businesses must establish timely and accurate security monitoring systems to ensure the safety of users and equipment.

[0003] However, current methods of elevator car safety monitoring often rely on manual inspection of video surveillance footage or detection through simple image recognition. These monitoring methods have many limitations. For example, the current centralized communication model requires all video data to be uploaded to a cloud server for processing, resulting in slow response times and an inability to provide timely warnings.

[0004] Furthermore, simple image recognition algorithms typically perform well only in well-lit experimental environments. Their performance drops drastically in real, complex environments. The lighting conditions inside elevator cars are complex, and images taken in insufficient lighting will appear dark and noisy. This will pose significant challenges to subsequent image recognition tasks and may even lead to warning failures. Summary of the Invention

[0005] The main objective of this application is to propose an edge-cloud collaborative elevator car safety monitoring method and related equipment, aiming to improve the efficiency of safety monitoring and adaptability to complex environments, realize all-weather monitoring and rapid response of elevator equipment and passenger safety, and issue timely warnings.

[0006] To achieve the above objectives, one aspect of this application proposes an edge-cloud collaborative elevator car safety monitoring system, the method comprising: Acquire visual images of the elevator car; The visual images are assessed for image quality and classified into normal lighting images and low lighting images. The low-light image is subjected to low-light enhancement processing to obtain a low-light enhanced image; Based on the low-light enhanced image, device safety classification and prediction are performed to obtain cloud computing results; Passenger safety monitoring is performed on the normal lighting image using a single-stage target detection algorithm to obtain edge calculation results; Security early warning monitoring is performed based on the edge computing results and the cloud computing results to obtain early warning results.

[0007] In some embodiments, performing low-light enhancement processing on the low-light image to obtain a low-light enhanced image includes the following steps: The low-light image is input into a pre-trained channel enhancement model for feature extraction and decoding to obtain a three-channel image enhancement matrix; Image enhancement is performed based on the low-light image and the three-channel image enhancement matrix to obtain a channel-enhanced image; The enhanced channel image is denoised to obtain a denoised image; The edge information of the denoised image is enhanced by Gaussian low-pass filtering to obtain a low-light enhanced image.

[0008] In some embodiments, the channel enhancement model is obtained through the following steps: The channel enhancement model is trained based on the training images to obtain the training enhancement images; Based on the color balance loss function, spatial consistency loss function, exposure loss function, and image smoothing loss function, the parameters of the channel enhancement model are optimized according to the training image and the training enhancement image to obtain a trained channel enhancement model.

[0009] In some embodiments, optimizing the parameters of the channel enhancement model based on the training image and the training enhanced image, using a color balance loss function, a spatial consistency loss function, an exposure loss function, and an image smoothing loss function, to obtain a trained channel enhancement model, includes the following steps: The color balance loss value is obtained by calculating the variance of the single-channel average pixel value and the three-channel average pixel value of the trained and enhanced image. The spatial consistency loss value is obtained by calculating the variance of the average pixel difference in the neighborhood of the training enhanced image and the average pixel difference in the neighborhood of the training image. The variance of the exposure intensity and the preset exposure of the training enhanced image is calculated to obtain the exposure loss value; Based on the three-channel image enhancement matrix corresponding to the trained enhanced image, the sum of the squares of the absolute values ​​of the horizontal and vertical gradients is accumulated to obtain the image smoothing loss value. The parameters of the channel enhancement model are optimized based on the color balance loss value, the spatial consistency loss value, the exposure loss value, and the image smoothing loss value to obtain a trained channel enhancement model.

[0010] In some embodiments, the step of performing image enhancement based on the low-light image and the three-channel image enhancement matrix to obtain a channel-enhanced image includes the following steps: The enhancement weights are obtained by adaptive weight calculation based on the original pixel values ​​of the low-light image; The original pixel value is non-linearly enhanced based on the enhancement weight and the three-channel image enhancement matrix to obtain the pixel enhancement value; Based on the pixel position relationship, a channel-enhanced image is obtained according to all the pixel enhancement values.

[0011] In some embodiments, the step of performing device security classification prediction based on the low-light enhanced image to obtain cloud computing results includes the following steps: The low-light enhancement image is subjected to feature extraction and analysis using a convolutional neural network to obtain local features; A fully connected network is used to perform binary classification prediction based on the local features to obtain a cloud computing result, wherein the cloud computing result indicates whether the device is normal or faulty.

[0012] In some embodiments, the step of performing passenger safety monitoring on the normal lighting image using a single-stage target detection algorithm to obtain edge calculation results includes the following steps: The normal lighting image is semantically enhanced to obtain a semantically enhanced image; Based on a multi-scale channel attention mechanism, multi-scale feature extraction and weight adjustment are performed on the semantically enhanced image to obtain a fused feature map. The fused feature map is input into a pre-trained intelligent target detection model to perform passenger counting, dangerous item detection, and passenger behavior recognition in parallel, and edge computing results are obtained.

[0013] To achieve the above objectives, another aspect of this application proposes an edge-cloud collaborative elevator car safety monitoring system, the system comprising: The first module is used to acquire visual images inside the elevator car; The second module is used to judge the image quality of the visual image and divide the visual image into normal lighting image and low lighting image; The third module is used to perform weak light enhancement processing on the low-light image to obtain a weak light enhanced image; The fourth module is used to perform device safety classification prediction based on the low-light enhanced image to obtain cloud computing results; The fifth module is used to perform passenger safety monitoring on the normal lighting image using a single-stage target detection algorithm to obtain edge calculation results; The sixth module is used to perform security early warning monitoring based on the edge computing results and the cloud computing results, and obtain early warning results.

[0014] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0015] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0016] The embodiments of this application include at least the following beneficial effects: This application provides an edge-cloud collaborative elevator car safety monitoring method and related equipment. This scheme acquires visual images inside the elevator car; performs image quality judgment on the visual images, classifying them into normal lighting images and low lighting images; performs weak light enhancement processing on the low lighting images to obtain weak light enhanced images; performs equipment safety classification prediction based on the weak light enhanced images to obtain cloud computing results; performs passenger safety monitoring on normal lighting images using a single-stage target detection algorithm to obtain edge computing results; and performs safety early warning monitoring based on the edge computing results and cloud computing results to obtain early warning results. This can improve the efficiency of safety monitoring and adaptability to complex environments, achieving all-weather monitoring and rapid response for elevator equipment and passenger safety, and timely issuing early warnings. Attached Figure Description

[0017] Figure 1 This is a flowchart of the edge-cloud collaborative elevator car safety monitoring method provided in the embodiments of this application; Figure 2 This is an architecture diagram of an edge-cloud integrated security monitoring system for complex environments provided in this application embodiment; Figure 3 This is a schematic diagram of the structure of the edge computing terminal provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the cloud computing terminal provided in the embodiments of this application; Figure 5 This is a schematic diagram of a low-light image provided in an embodiment of this application; Figure 6 This is a schematic diagram of the enhanced image provided in the embodiments of this application; Figure 7 This is a schematic diagram of the structure of the computing module provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of the edge-cloud collaborative elevator car safety monitoring system provided in this application embodiment; Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of systems and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0020] Before providing a detailed description of the embodiments of this application, some related technologies involved in the embodiments of this application will be described first.

[0021] Intelligent video surveillance systems employ image processing, pattern recognition, and computer vision technologies. By adding intelligent video analysis modules to the surveillance system, and leveraging the powerful data processing capabilities of computers, they filter out useless or interfering information from video footage, automatically identify different objects, analyze and extract key and useful information from the video source, quickly and accurately locate accident scenes, determine abnormal situations in the surveillance footage, and issue alarms or trigger other actions in the fastest and best way. This enables effective pre-event warnings, in-event handling, and timely post-event evidence collection.

[0022] However, mainstream intelligent monitoring systems adopt a direct "end-to-cloud" communication mode. The entire process of data transmission, queuing, and return takes several seconds. For security monitoring scenarios that require second-level response, it is too late to issue an early warning by then.

[0023] Furthermore, current image recognition algorithms are weak in resisting interference factors such as noise and changes in lighting, resulting in low reliability in actual security monitoring. Manual inspection is still required to ensure the safety of users and equipment.

[0024] In view of this, this application provides an edge-cloud collaborative elevator car safety monitoring method and related equipment. This method acquires visual images inside the elevator car; performs image quality assessment on the visual images, classifying them into normal lighting images and low lighting images; performs weak light enhancement processing on the low lighting images to obtain weak light enhanced images; performs equipment safety classification prediction based on the weak light enhanced images to obtain cloud computing results; performs passenger safety monitoring on the normal lighting images using a single-stage target detection algorithm to obtain edge computing results; and performs safety early warning monitoring based on the edge computing results and cloud computing results to obtain early warning results. This improves the efficiency of safety monitoring and adaptability to complex environments, enabling 24 / 7 monitoring and rapid response for elevator equipment and passenger safety, and timely early warning.

[0025] The edge-cloud collaborative elevator car safety monitoring method provided in this application relates to the field of elevator safety technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the edge-cloud collaborative elevator car safety monitoring method, but is not limited to the above forms.

[0026] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0027] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0028] Figure 1 This is an optional flowchart of the edge-cloud collaborative elevator car safety monitoring method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S106.

[0029] Step S101: Obtain visual images inside the elevator car.

[0030] Step S102: Perform image quality judgment on the visual images and divide the visual images into normal lighting images and low lighting images.

[0031] Step S103: Perform low-light enhancement processing on the low-light image to obtain a low-light enhanced image.

[0032] Step S104: Perform device safety classification prediction based on the low-light enhanced image to obtain the cloud computing result.

[0033] Step S105: Passenger safety monitoring is performed on the normal lighting image using a single-stage target detection algorithm to obtain edge calculation results.

[0034] Step S106: Perform security early warning monitoring based on edge computing results and cloud computing results to obtain early warning results.

[0035] In this embodiment, to enable safety monitoring of the elevator car and passengers, an Intelligent Controlled Monitoring System (ICCTV) with an edge-cloud collaborative architecture is used. This system achieves 24 / 7 monitoring and rapid response for elevator equipment and passenger safety, greatly improving the efficiency of the safety monitoring system. The system employs a lightweight algorithm deployment based on the edge-cloud model. The algorithms include an intelligent target detection algorithm for the elevator car and a data augmentation algorithm for low-light environments. By deploying the intelligent target detection algorithm on the edge computing terminal and the data augmentation algorithm on the cloud computing terminal, flexible algorithm invocation and efficient operation are achieved.

[0036] Specifically, data acquisition is first completed by continuously collecting raw visual images through vision sensors deployed inside the car.

[0037] After acquiring a visual image, the system does not immediately perform image recognition. Instead, it first performs image quality judgment by judging factors such as the overall illumination level, contrast, and noise of the image. By calculating the global average pixel value of the image, the system adaptively divides the visual image into normal illumination images and low illumination images.

[0038] Normal lighting images are visual images whose overall brightness statistics and signal-to-noise ratio are both higher than the preset judgment threshold, and can be directly input into the target detection algorithm deployed on the edge computing terminal.

[0039] Low-light images are visual images where the overall brightness is concentrated in a low range and the signal-to-noise ratio is significantly lower than the judgment threshold. Such images appear dark, with blurred details and significant noise. If target detection is performed directly, the reduced recognition performance will cause great difficulties for subsequent image recognition tasks and may even lead to warning failure. Therefore, it is necessary to perform data augmentation on low-light images first. For algorithms with high accuracy requirements, the number of parameters and inference speed of the data augmentation algorithm should be fully considered, and a cloud computing terminal should be designed for algorithm deployment.

[0040] Furthermore, for visual images identified as low-light images, unlike supervised low-light intensity algorithms, this application employs a linearly arranged network structure design to ensure lightweight operation and minimize system load. The low-light enhancement algorithm collaboratively optimizes the image's brightness, color, and noise, ultimately outputting a low-light enhanced image with clear details, which can be used for advanced semantic analysis.

[0041] Subsequently, device safety classification prediction is performed based on the low-light enhanced image. The input low-light enhanced image undergoes feature extraction and analysis to determine whether the device has defects, and cloud computing results characterizing the device status are obtained.

[0042] For visual images determined to be under normal lighting conditions, processing is performed directly at the edge computing terminal. The edge computing terminal integrates a single-stage object detection algorithm, which can directly predict the category and location of the target, eliminating the step of generating candidate regions in the two-stage algorithm, thereby achieving faster detection speed. Real-time object detection can be performed after data acquisition, improving the efficiency of the overall architecture. Based on the single-stage object detection algorithm, the safety monitoring system can identify and locate passengers, foreign objects, and specific behaviors, thereby obtaining edge computing results that characterize the passenger safety status.

[0043] Finally, the system enters the fusion decision-making stage, which aggregates and analyzes the cloud computing results returned by the cloud computing terminal and the edge computing results returned by the edge computing terminal to comprehensively assess the overall safety situation inside the elevator car. When any potential or actual risk is identified, different levels of early warning information are immediately generated and the corresponding early warning notification mechanism is triggered, thereby completing the closed loop from perception and analysis to decision-making and early warning, realizing all-weather monitoring and rapid response for elevator equipment and passenger safety, and issuing timely warnings.

[0044] In some embodiments, step S103 may include, but is not limited to, steps S201 to S204.

[0045] Step S201: Input the low-light image into the pre-trained channel enhancement model for feature extraction and decoding to obtain a three-channel image enhancement matrix.

[0046] Step S202: Perform image enhancement based on the low-light image and the three-channel image enhancement matrix to obtain a channel-enhanced image.

[0047] Step S203: Denoise the channel-enhanced image to obtain a denoised image.

[0048] Step S204: Based on the edge information of the Gaussian low-pass filter, the weak light enhanced image is obtained.

[0049] In this embodiment, the low-light enhancement algorithm consists of three stages. The first stage is an enhancement method based on the RGB channel curves of the image. This stage enhances low-quality low-light images by learning the features of a large number of low-light images. During feature extraction and decoding, the trained channel enhancement model uses multi-layer depthwise separable convolution operations to finally calculate the three-channel enhancement matrix. Specifically, the low-light image is input into the pre-trained channel enhancement model. The encoder part parses the key visual information of dark areas and noise from the low-quality input and encodes it into high-dimensional features. Then, the decoder parses the high-dimensional features to obtain a three-channel image enhancement matrix for repairing the low-light image. The three-channel image enhancement matrix is ​​a pixel-level instruction set output by the channel enhancement model according to the content of the low-light image. The three channels refer to the most basic red (R), green (G), and blue (B) color channels of the digital image. The R channel records the red intensity of each pixel, the G channel records the green intensity of each pixel, and the B channel records the blue intensity of each pixel. The three-channel image enhancement matrix records how many times the red, green, and blue channel components of each pixel in the low-light image need to be enhanced.

[0050] After obtaining the three-channel image enhancement matrix, the three-channel image enhancement matrix is ​​combined with the original low-light image for image enhancement.

[0051] Specifically, the three-channel image enhancement matrix contains enhancement coefficients for each of the three channels. Each pixel in each channel has a corresponding enhancement coefficient. For example, the enhancement coefficient for pixel 1 in the R channel is 2, and the enhancement coefficient for pixel 2 in the R channel is 1.3. The enhancement coefficients for each pixel in each channel are independent of each other. Considering that if the three-channel image enhancement matrix is ​​simply linearly multiplied with a low-light image, for example, assuming the R channel brightness value of pixel 1 is R1=0.3 and the R channel brightness value of pixel 2 is R2=0.9, if directly multiplied linearly, the image enhancement results in R1=0.6 and R2=1.17. However, in digital images, pixel brightness is usually limited to between 0 and 1, and 1.17 exceeds the upper limit, causing the pixel to be severely overexposed and lose details after image enhancement.

[0052] In view of this, this embodiment adaptively adjusts the enhancement coefficient using a non-linear transformation based on the brightness of each pixel in the low-light image. If the current pixel is already within the normal exposure range, a smaller enhancement weight is used to adjust the enhancement coefficient; conversely, if the current pixel is in a dark area, a larger enhancement weight is used. By using an intelligent image enhancement algorithm to enhance low-light images, it can effectively brighten dark areas caused by insufficient light and prevent overexposure in bright areas, resulting in a channel-enhanced image.

[0053] The second stage is image denoising. Although the signal-to-noise ratio of the low-light image has been improved after the first stage of image enhancement, the inherent noise in the original image is also amplified during the enhancement process due to its low quality. Therefore, special denoising processing is required for the channel-enhanced image.

[0054] For example, a deep learning network such as UNet is used to denoise the channel-enhanced image. Through the symmetrical structure and skip connections of the UNet network, the network can extract features at different resolutions, thereby improving the denoising effect and obtaining a denoised image.

[0055] Building upon noise suppression, Gaussian low-pass filtering is used to enhance the edge information of the denoised image in order to improve its semantic information. By selectively enhancing specific spatial frequency components in the denoised image, low-frequency information is separated and processed, and then weighted and fused with the denoised image to strengthen the semantic information.

[0056] Specifically, a Gaussian blur low-pass filter is first used to separate and extract low-frequency information from the input data (i.e., the denoised image after three-channel enhancement and denoising), effectively preserving the overall contour and brightness distribution of the image while reducing the interference of high-frequency noise. Subsequently, the low-frequency information is interactively processed with the denoised image to enhance and compensate for high frequencies such as edges and textures in the denoised image, so that the final generated image can maintain clear details and a natural visual effect while removing noise.

[0057] The calculation formula for the interaction between low-frequency components and the denoised image is as follows: (1); in, E img To enhance images in low light, D img For input data, LF img For low-frequency information in the input data, α and β These are empirical parameters; in the experiment, they are determined based on experience. α Set to 1.5. β Set it to 0.5.

[0058] In some embodiments, the channel enhancement model is obtained through steps S301 to S302.

[0059] Step S301: Train the channel enhancement model based on the training image to obtain the training enhancement image.

[0060] Step S302: Based on the color balance loss function, spatial consistency loss function, exposure loss function, and image smoothing loss function, the parameters of the channel enhancement model are optimized according to the training image and the training enhancement image to obtain the trained channel enhancement model.

[0061] In this embodiment, the training images include several pairs of low-light reference images and their corresponding normal-light reference images. The low-light reference images are input into the channel enhancement model to be trained. The model sequentially completes the process of feature extraction, decoding and generating a three-channel image enhancement matrix based on the model's internal parameters. The low-light images are enhanced using the three-channel image enhancement matrix generated by the channel enhancement model, and the training enhanced images predicted by the channel enhancement model can be obtained.

[0062] After obtaining the training augmented image, the parameters of the channel augmentation model are optimized based on the training augmented image and the normal lighting reference image in the training image. Through continuous iterative updates of massive training data, the model parameters converge to a stable state, resulting in a well-trained channel augmentation model.

[0063] Specifically, this embodiment guides the optimization of model training through four composite loss functions and enhances the model's channel enhancement capability through multi-dimensional evaluation criteria.

[0064] First is the color balance loss function, which is used to constrain the average value of the enhanced image across all color channels to be consistent, thus penalizing any potential color distortion.

[0065] For example, if an environment that would normally appear yellowish under dim lighting becomes bluish after channel enhancement, the color balance loss function will produce a high loss value to correct the color mapping relationship in subsequent iterations of the channel enhancement model.

[0066] Secondly, there is the spatial consistency loss function, which is used to maintain the brightness and darkness relationships in the local neighborhood of the image.

[0067] For example, the original low-light training image is divided into multiple local regions according to a preset area, and the brightness difference between the local region and its four adjacent regions (top, bottom, left, and right) before and after enhancement is calculated. For example, in a face image, assuming the image in the current local region is the forehead, and the image in the local region below this local region is the cheek, in the original low-light training image, the brightness of the local region where the forehead is located is greater than that of the local region where the cheek is located. After channel enhancement, the training enhanced image should still ensure that the brightness of the local region where the forehead is located is greater than that of the local region where the cheek is located. If channel enhancement causes a change in the brightness relationship of the local neighborhood, the spatial consistency loss function will produce a high loss value to correct the distortion of the brightness relationship in subsequent iterations of the channel enhancement model.

[0068] Next is the exposure loss function, which is used to push the average brightness of local areas closer to the ideal empirical value, thereby avoiding the output training enhancement image from being too dark overall or overexposed in large areas.

[0069] For example, the ideal exposure is set to 0.5 based on experience. If the exposure after channel enhancement is much less than or much greater than the ideal exposure, the exposure loss function will produce a high loss value to correct local or global underexposure and overexposure in subsequent iterations of the channel enhancement model.

[0070] Finally, there is the image smoothing loss function, which is used to ensure that the gradient changes of the generated three-channel image enhancement matrix in the horizontal and vertical directions are smoothly transitioned.

[0071] For example, when the enhancement coefficient of an object edge in the training enhanced image jumps, such as from 1.2 to 3.5, or when the enhancement matrices of the R, G, and B channels produce different changes in the same region, such as the R channel having a large gradient in the horizontal direction and the B channel having a large gradient in the vertical direction, resulting in color separation, the image smoothing loss function will produce a high error value to correct the drastic changes in the matrix during subsequent iterations of the channel enhancement model, thereby suppressing possible boundary defects.

[0072] By working together through the four loss functions, the four loss values ​​are weighted and merged to obtain the total loss of the channel enhancement model. The model parameters are then iteratively updated to finally obtain the trained channel enhancement model.

[0073] In some embodiments, step S302 may include, but is not limited to, steps S401 to S405.

[0074] Step S401: Calculate the variance based on the single-channel average pixel value and the three-channel average pixel value of the training enhancement image to obtain the color balance loss value.

[0075] Step S402: Calculate the variance based on the average pixel difference between the neighborhood of the enhanced image and the average pixel difference between the neighborhood of the training image to obtain the spatial consistency loss value.

[0076] Step S403: Calculate the variance based on the training enhanced image and the preset exposure to obtain the exposure loss value.

[0077] Step S404: Based on the three-channel image enhancement matrix corresponding to the training enhanced image, the sum of the squares of the absolute values ​​of the horizontal and vertical gradients is accumulated to calculate the image smoothing loss value.

[0078] Step S405: Optimize the parameters of the channel enhancement model based on the color balance loss value, spatial consistency loss value, exposure loss value, and image smoothing loss value to obtain a trained channel enhancement model.

[0079] In this embodiment, channel pixel values ​​are used to represent the brightness or intensity of a pixel in a specific color channel of an image, where 0 represents the darkest and 255 represents the brightest. The average of all channel pixel values ​​for a given color channel is the single-channel average pixel value. For example, in a 2x2 pixel image, if its green channel pixels are [[100,150],[50,200]], then the single-channel average pixel value for the green channel is 125. The average of the single-channel average pixel values ​​corresponding to the three color channels for a given pixel is the three-channel average pixel value.

[0080] Optionally, for ease of calculation, the pixel values ​​can be normalized to map all pixel values ​​to the range [0,1].

[0081] Calculate the average pixel values ​​of the red, green, and blue channels in the training and enhancement image. Then, calculate the variance using the color balance loss function based on the average pixel values ​​of the red, green, and blue channels. The expression for the color balance loss function is as follows: (2); in, L col It is the color balance loss value. N It is the number of channels in the training and enhancement image. I i is the single-channel average pixel value, representing the average pixel value of each channel in the training enhancement image. M is the three-channel average pixel value, representing the average of the pixel values ​​of the three channels, calculated using the following formula: (3); in, I R It is the average pixel value of a single channel in the red channel. I G It is the average pixel value of a single channel in the green channel. I B It is the average pixel value of the blue channel.

[0082] The training enhancement image is divided into several local regions according to the preset region size. The average pixel value of all pixels and all channels in each local region is calculated to obtain the region average pixel value. Based on the local region in the training enhancement image, the difference between its average pixel value and the average pixel value of its four neighboring regions is calculated to obtain the corresponding neighborhood average pixel difference value of the training enhancement image.

[0083] Similarly, based on the local region in the low-light reference image of the training image that corresponds to the training enhancement image, the difference between the average pixel value of the region and the average pixel value of the region in the four neighboring regions is calculated to obtain the average pixel difference of the neighborhood corresponding to the training image.

[0084] The spatial consistency loss function calculates the variance based on the average pixel difference between the neighborhoods of the training and augmented images. The expression for the spatial consistency loss function is as follows: (4); in, L spa It is the spatial consistency loss value. M It is the number of regions in the training enhancement image. Ω Is with the first jThe four adjacent regions are defined as follows: j is the number of the four regions. X i It is the first in the training enhancement image i th The average pixel value of each region Y i It is the average pixel value of the i-th region in the low-light reference image.

[0085] The exposure loss function calculates the variance based on the exposure intensity of the training enhanced image and the preset exposure. The expression for the exposure loss function is as follows: (5); in, L exp It is the exposure loss value. M It is the number of regions in the training enhancement image. I r To enhance the numerical intensity in region r after the image is converted to grayscale, Z To preset the exposure, in the experiment, Z Set it to 0.5.

[0086] The image smoothing loss is applied to the three-channel image enhancement matrix generated by the model. This loss is calculated using the image smoothing loss function, which calculates the square of the sum of the absolute values ​​of the horizontal and vertical gradients based on the three-channel image enhancement matrix corresponding to the training enhanced image, and then sums these values ​​across all locations. The expression for the image smoothing loss function is: (6); in, L tvA This is the exposure loss value, ▽ w and ▽ h These are horizontal and vertical gradient operations, respectively. A c It is the enhancement matrix for each channel corresponding to the training enhanced image.

[0087] After obtaining the loss values ​​for the four dimensions mentioned above, the values ​​are then weighted according to predetermined weighting coefficients. L col , L spa , L exp , L tvA A weighted summation is performed to obtain a total loss value. Then, based on the backpropagation algorithm of the error, the total loss value is used to iteratively update the model parameters through optimization algorithms such as gradient descent. When the iteration reaches the maximum number of times or converges, the model parameters generated in the last iteration are used as the internal parameters of the channel enhancement model, and the trained channel enhancement model is obtained.

[0088] In some embodiments, step S202 may include, but is not limited to, steps S501 to S503.

[0089] Step S501: Adaptive weight calculation is performed based on the original pixel values ​​of the low-light image to obtain the enhancement weight.

[0090] Step S502: Perform nonlinear enhancement on the original pixel values ​​according to the enhancement weights and the three-channel image enhancement matrix to obtain pixel enhancement values.

[0091] Step S503: Based on the pixel position relationship, obtain the channel-enhanced image according to the enhancement values ​​of all pixels.

[0092] In this embodiment, to achieve the nonlinear transformation of channel enhancement, the original pixel value of each pixel is first normalized, and then the dedicated enhancement weight is dynamically calculated.

[0093] For example, assuming that the sum of the enhancement weight and the normalized original pixel value is always 1, for an original pixel value, if the normalized value is 0.3, the corresponding enhancement weight is 0.7, and if the normalized value is 0.8, the corresponding enhancement weight is 0.2.

[0094] By establishing an inverse relationship between weight and brightness, overexposure in highlight areas can be avoided, while shadow areas can be enhanced.

[0095] Alternatively, pixels that are close to pure black or pure white can be given a lower weight, while pixels in the middle brightness range can be given a higher weight, thereby enhancing shadow details while effectively preventing excessive amplification of noise and overexposure of highlight areas.

[0096] After obtaining the enhancement weight of each pixel, it is combined with the coefficients at the corresponding positions in the three-channel image enhancement matrix. The two together determine the pixel enhancement value. For example, for a pixel with a normalized original pixel value of 0.4, if the coefficients corresponding to its enhancement matrix are 1.5 and the calculated enhancement weight is 0.6, then the pixel enhancement value is 0.4*1.5*0.6=0.36.

[0097] By superimposing the pixel enhancement value onto the original pixel value, the non-linearly enhanced pixel value is obtained as 0.4 + 0.36 = 0.76.

[0098] In summary, to simplify the nonlinear enhancement process, the following formula is used: (7); in, E It is a three-channel image enhancement matrix. I inIt is the pixel value after the values ​​of the three RGB channels of the captured image are normalized to the range of 0 to 1.

[0099] After completing the independent enhancement of all pixels, the original pixel values ​​after superimposing the pixel enhancement values ​​are recombined based on the inherent pixel position relationship to restore the spatially coherent two-dimensional digital image, thus obtaining the channel-enhanced image.

[0100] In some embodiments, step S104 may include, but is not limited to, steps S601 to S602.

[0101] Step S601: Use a convolutional neural network to extract and analyze features from the low-light enhancement image to obtain local features.

[0102] Step S602: Use a fully connected network to perform binary classification prediction based on local features to obtain the cloud computing result, where the cloud computing result indicates whether the device is normal or faulty.

[0103] In this embodiment, based on the enhanced low-light image, a convolutional neural network is used to perform convolution calculations. The input image data undergoes feature extraction and analysis. The initial convolutional layer is responsible for capturing features such as edges and textures in the image. As the network layers deepen, subsequent convolutional layers further combine more complex and higher-order local features based on the above features. These local features correspond to the device's component shape, structural deformation, and other content, providing a data basis for subsequent classification decisions.

[0104] After the convolutional neural network outputs local features, a fully connected network is used to complete the classification prediction. The fully connected network comprehensively considers all the extracted local features and learns the mapping relationship between them and device security. The entire classification process is a binary classification, that is, the final output cloud computing result is 0 or 1. Optionally, 0 represents "device normal" and 1 represents "device malfunction".

[0105] The output cloud computing results will be sent to the client to provide maintenance personnel with equipment safety assessment and decision-making basis, thereby realizing intelligent diagnosis and monitoring of elevator equipment.

[0106] In some embodiments, step S105 may include, but is not limited to, steps S701 to S703.

[0107] Step S701: Perform semantic enhancement on the normal illumination image to obtain a semantically enhanced image.

[0108] Step S702: Based on the multi-scale channel attention mechanism, multi-scale feature extraction and weight adjustment are performed on the semantically enhanced image to obtain a fused feature map.

[0109] Step S703: Input the fused feature map into the pre-trained intelligent target detection model to perform passenger counting, dangerous item detection and passenger behavior recognition in parallel to obtain edge computing results.

[0110] In this embodiment, in order to improve the semantic information expression and the effectiveness of multi-level features of normal lighting images, the single-stage target detection algorithm adopts Gaussian blur low-pass filtering and channel attention mechanism to complete the identification of specific objects at the edge computing terminal.

[0111] Specifically, similar to step S203, edge information of the normal illumination image is enhanced based on Gaussian low-pass filtering, smoothing the fine textures in the normal illumination image, preserving and highlighting the overall shape and boundaries of objects, such as making the outline of a passenger's limbs, the shape of an electric vehicle, and the boundaries of various items clearer and more prominent, thereby obtaining a semantically enhanced image to improve the robustness of subsequent tasks.

[0112] Subsequently, the semantically enhanced image is input into a feature extraction network integrating a multi-scale channel attention mechanism. This network first extracts feature maps at different scales from the semantically enhanced image, and then uses the channel attention mechanism to adjust the weights of the multi-scale feature maps. It adaptively evaluates the importance of each feature channel at different scales and assigns them appropriate weights. A fused feature map is obtained by fusing the weighted multi-scale features.

[0113] Finally, the fused feature map is input into an intelligent object detection model pre-trained using deep learning and corresponding datasets. This model simultaneously counts the number of bounding boxes identified as "people," identifies specific categories such as "electric vehicles" and "knives," and analyzes the pose of human bounding boxes. All recognition results are then integrated into edge computing results and uploaded to the system's decision center to enable perception and early warning of passenger safety within the elevator car.

[0114] The following is a detailed description and explanation of the solutions in the embodiments of the present invention, using specific application examples: In order to enable safety monitoring of equipment and passengers inside the elevator car, refer to Figure 2 , Figure 2 This is an architecture diagram of an edge-cloud integrated security monitoring system for complex environments provided in this application embodiment. As can be seen from the diagram, the system mainly includes the following components: an image acquisition module, an edge computing terminal, a cloud computing terminal, and an early warning system.

[0115] The image acquisition module contains several vision sensors deployed in different elevator cars to capture images of the car interiors in real time. All acquired visual images first pass through an image quality assessment module, which classifies them into low-light images and normal-light images based on factors such as brightness and noise.

[0116] The cloud computing terminal receives low-light images for device safety analysis and ultimately outputs the cloud computing results to the early warning system.

[0117] The edge computing terminal receives images of normal lighting conditions to perform passenger safety analysis, and finally outputs the edge computing results to the early warning system.

[0118] Specifically, the design of this edge-cloud integrated security monitoring system includes the following stages: (I) Edge Cloud Model Deployment Phase: (1) Design of edge computing terminal and cloud computing terminal.

[0119] To achieve the goal of deploying computing on the visual sensor side, this embodiment designs an edge computing terminal. Based on the AI ​​chip computing unit, a single-stage detection algorithm is designed to realize real-time object detection after data acquisition, thereby improving the efficiency of the overall architecture.

[0120] Specifically, such as Figure 3 As shown, the edge computing terminal comprises three main modules: a power supply module, a computing module, and a data transmission module. The data transmission module is responsible for data interaction between the various parts; the computing module is responsible for processing the acquired visual images; and the power supply module is responsible for supplying power to the edge computing module.

[0121] The power supply module has three interfaces: one is the power input for the edge computing terminal, one is the power output for the external vision sensor, and the last is the power output for the computing module. The power supply module ensures the stable operation of the entire system.

[0122] The data transmission module is responsible for data interaction between the edge computing terminal and external modules. This data interaction includes interaction between the computing module and sensors, as well as the outward transmission of data.

[0123] The computing module is the core component of the edge computing terminal, and its main function is to process images acquired by sensors in real time. The computing module is equipped with an AI chip, which allows algorithms to be directly imported into it, enabling remote data processing.

[0124] Furthermore, for algorithms with high accuracy requirements, this embodiment fully considers the number of parameters and inference speed of the data augmentation algorithm, and also designs a cloud computing terminal for algorithm deployment. By deploying the in-car detection algorithm on the edge computing terminal and the data augmentation algorithm on the cloud computing terminal, flexible invocation and efficient operation of the algorithms are achieved.

[0125] (2) Design of data transmission mode and client entity.

[0126] Considering that data transmission should adhere to the principles of low cost and high efficiency, a four-step strategy was developed to determine the data transmission protocol. The first step is defining the application scenario, including analyzing the scenario and summarizing its characteristics, to provide a sufficient basis for subsequent solution determination.

[0127] The next step is to choose the transmission mode. The appropriate transmission mode should be selected based on the specific scale of the application scenario. Transmission methods include wired and wireless. For terminals with short distances and ample space, wired connections can be chosen. For remote or dynamic scenarios, wireless transmission can be selected for communication between terminals.

[0128] The next step is to evaluate the transmission mode. After determining the initial data transmission method, the transmission scheme is evaluated. From the perspective of system load and cost, the data transmission method is evaluated and optimized accordingly to ensure accuracy and minimize costs.

[0129] The final step is to determine the transmission nodes. Based on the transmission method and application scenario, analyze the possible transmission nodes and set up corresponding bridging network points to ensure smooth communication across the overall architecture.

[0130] For the client, mature cloud servers are used as data terminals and data storage devices. The data on the server is divided into real-time data and historical data. When data needs to be sent to the client, some data is directly pushed to the front end for display, while the other part of the data is first analyzed by the processing unit, and the results are sent to the front end.

[0131] (II) Algorithm Design Stage: When the application scene is poorly lit, the captured images will appear dark and noisy. Therefore, this embodiment designs an image enhancement module as a cloud computing terminal. First, a lightweight network structure is designed, and an adaptive image enhancement matrix learning module based on image feature blocks is constructed to enhance low-quality images.

[0132] Specifically, refer to Figure 4 The adaptive image enhancement matrix learning module consists of a feature extraction module and an enhancement module.

[0133] First, high-dimensional features are extracted from the acquired low-light image based on the feature extraction module, and a three-channel image enhancement matrix is ​​obtained through the decoder. Then, the enhancement module uses formula (7) to enhance the low-light image according to the three-channel image enhancement matrix. The enhancement module is supervised by four loss functions.

[0134] Furthermore, after enhancement using a three-channel image enhancement matrix, the image is denoised. Simultaneously, to enhance the semantic information of the image, Gaussian low-pass filtering is used to enhance the image's edge information.

[0135] By using the enhancement module, low-light images with low light and noise can be enhanced and denoised to improve the robustness of subsequent tasks.

[0136] Finally, based on the enhanced images, a classification module is designed to complete the binary classification problem of safety monitoring. The image data input to the classification module is processed by feature extraction and analysis to determine whether there are defects in the workpiece. The classification module uses a convolutional neural network for feature extraction and then uses a fully connected network to complete the classification prediction.

[0137] For example, refer to Figure 5 , Figure 5 The low-light image provided in this application, after being enhanced by the image enhancement method of this application, is as follows: Figure 6 As shown.

[0138] Intelligent object detection algorithms need to be deployed in the computing modules of edge computing terminals to identify specific objects through a designed single-stage object detection algorithm.

[0139] like Figure 7 As shown, the acquired normal lighting image is first input into the semantic enhancement module. The semantic enhancement module has the same purpose as the aforementioned low light enhancement algorithm: to enhance the edge information of the image.

[0140] Next, the semantically enhanced normal lighting image is input into the multi-scale feature extraction module, which is a pyramid convolutional neural network that can extract feature blocks of different scales.

[0141] The channel attention module adjusts the weights of multi-scale feature blocks and assigns values ​​to different channels.

[0142] Finally, the detector performs target detection on the output of the channel attention module to identify illegal and dangerous behaviors such as electric vehicles and people falling.

[0143] In summary, this embodiment uses elevator equipment and passengers as monitoring targets, achieving low-latency and high-precision safety monitoring performance. The safety monitoring system includes a data augmentation algorithm designed for low-light environments and an intelligent target detection algorithm for the elevator car. To ensure efficient operation of the intelligent algorithms, a lightweight algorithm deployment based on an edge-cloud model was implemented, enabling 24 / 7 monitoring and rapid response for elevator equipment and passenger safety, greatly improving the efficiency of the safety monitoring system.

[0144] Reference Figure 8This application also provides an edge-cloud collaborative elevator car safety monitoring system that can implement the above-mentioned method. The system includes: The first module is used to acquire visual images inside the elevator car.

[0145] The second module is used to judge the image quality of visual images, classifying them into normal lighting images and low lighting images.

[0146] The third module is used to perform low-light enhancement processing on low-light images to obtain low-light enhanced images.

[0147] The fourth module is used to perform device safety classification prediction based on low-light enhanced images and obtain cloud computing results.

[0148] The fifth module is used to perform passenger safety monitoring on normally lit images using a single-stage target detection algorithm and obtain edge calculation results.

[0149] The sixth module is used to perform security early warning monitoring based on edge computing and cloud computing results, and obtain early warning results.

[0150] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0151] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0152] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0153] Reference Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0154] The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the methods described in the embodiments of this application.

[0155] The input / output interface 903 is used to implement information input and output.

[0156] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0157] Bus 905 transmits information between various components of the device, such as processor 901, memory 902, input / output interface 903, and communication interface 904.

[0158] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0159] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0160] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0161] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0162] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0163] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0164] The edge-cloud collaborative elevator car safety monitoring method and related equipment provided in this application embodiment acquire visual images inside the elevator car; perform image quality judgment on the visual images, classifying them into normal lighting images and low lighting images; perform weak light enhancement processing on the low lighting images to obtain weak light enhanced images; perform equipment safety classification prediction based on the weak light enhanced images to obtain cloud computing results; perform passenger safety monitoring on normal lighting images using a single-stage target detection algorithm to obtain edge computing results; and perform safety early warning monitoring based on the edge computing results and cloud computing results to obtain early warning results. This method can improve the efficiency of safety monitoring and adaptability to complex environments, enabling all-weather monitoring and rapid response for elevator equipment and passenger safety, and timely early warning.

[0165] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0166] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0167] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0168] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0169] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0170] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0171] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0172] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0173] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0174] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0175] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for edge-cloud collaborative elevator car safety monitoring, characterized in that, The method Includes the following steps; Acquire visual images of the elevator car; The visual images are assessed for image quality and classified into normal lighting images and low lighting images. The low-light image is subjected to low-light enhancement processing to obtain a low-light enhanced image; Based on the low-light enhanced image, device safety classification and prediction are performed to obtain cloud computing results; Passenger safety monitoring is performed on the normal lighting image using a single-stage target detection algorithm to obtain edge calculation results; Security early warning monitoring is performed based on the edge computing results and the cloud computing results to obtain early warning results.

2. The method according to claim 1, characterized in that, The process of performing weak light enhancement processing on the low-light image to obtain a weak light enhanced image includes the following steps: The low-light image is input into a pre-trained channel enhancement model for feature extraction and decoding to obtain a three-channel image enhancement matrix; Image enhancement is performed based on the low-light image and the three-channel image enhancement matrix to obtain a channel-enhanced image; The enhanced channel image is denoised to obtain a denoised image; The edge information of the denoised image is enhanced by Gaussian low-pass filtering to obtain a low-light enhanced image.

3. The method according to claim 2, characterized in that, The channel enhancement model is obtained through the following steps: The channel enhancement model is trained based on the training images to obtain the training enhancement images; Based on the color balance loss function, spatial consistency loss function, exposure loss function, and image smoothing loss function, the parameters of the channel enhancement model are optimized according to the training image and the training enhancement image to obtain a trained channel enhancement model.

4. The method according to claim 3, characterized in that, The process of optimizing the parameters of the channel enhancement model based on the color balance loss function, spatial consistency loss function, exposure loss function, and image smoothing loss function, using the training image and the training enhanced image to obtain a trained channel enhancement model, includes the following steps: The color balance loss value is obtained by calculating the variance of the single-channel average pixel value and the three-channel average pixel value of the trained and enhanced image. The spatial consistency loss value is obtained by calculating the variance of the average pixel difference in the neighborhood of the training enhanced image and the average pixel difference in the neighborhood of the training image. The variance of the exposure intensity and the preset exposure of the training enhanced image is calculated to obtain the exposure loss value; Based on the three-channel image enhancement matrix corresponding to the trained enhanced image, the sum of the squares of the absolute values ​​of the horizontal and vertical gradients is accumulated to obtain the image smoothing loss value. The parameters of the channel enhancement model are optimized based on the color balance loss value, the spatial consistency loss value, the exposure loss value, and the image smoothing loss value to obtain a trained channel enhancement model.

5. The method according to claim 2, characterized in that, The step of performing image enhancement based on the low-light image and the three-channel image enhancement matrix to obtain a channel-enhanced image includes the following steps: The enhancement weights are obtained by adaptive weight calculation based on the original pixel values ​​of the low-light image; The original pixel value is non-linearly enhanced based on the enhancement weight and the three-channel image enhancement matrix to obtain the pixel enhancement value; Based on the pixel position relationship, a channel-enhanced image is obtained according to all the pixel enhancement values.

6. The method according to claim 1, characterized in that, The process of performing device security classification and prediction based on the low-light enhanced image to obtain cloud computing results includes the following steps: The low-light enhancement image is subjected to feature extraction and analysis using a convolutional neural network to obtain local features; A fully connected network is used to perform binary classification prediction based on the local features to obtain a cloud computing result, wherein the cloud computing result indicates whether the device is normal or faulty.

7. The method according to claim 1, characterized in that, The process of performing passenger safety monitoring on the normal lighting image using a single-stage target detection algorithm to obtain edge calculation results includes the following steps: The normal lighting image is semantically enhanced to obtain a semantically enhanced image; Based on a multi-scale channel attention mechanism, multi-scale feature extraction and weight adjustment are performed on the semantically enhanced image to obtain a fused feature map. The fused feature map is input into a pre-trained intelligent target detection model to perform passenger counting, dangerous item detection, and passenger behavior recognition in parallel, and edge computing results are obtained.

8. An edge-cloud collaborative elevator car safety monitoring system, characterized in that, The system includes: The first module is used to acquire visual images inside the elevator car; The second module is used to judge the image quality of the visual image and divide the visual image into normal lighting image and low lighting image; The third module is used to perform weak light enhancement processing on the low-light image to obtain a weak light enhanced image; The fourth module is used to perform device safety classification prediction based on the low-light enhanced image to obtain cloud computing results; The fifth module is used to perform passenger safety monitoring on the normal lighting image using a single-stage target detection algorithm to obtain edge calculation results; The sixth module is used to perform security early warning monitoring based on the edge computing results and the cloud computing results, and obtain early warning results.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images