Equipment management method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses an equipment management method and device, equipment and a storage medium, which can be applied to the field of image processing, and the method comprises the following steps: obtaining an infrared thermal image of target equipment, and carrying out image partitioning processing on the infrared thermal image to obtain a plurality of sub-images; inputting each sub-image of the infrared thermal image into the temperature prediction model to obtain a predicted temperature value of the infrared thermal image; managing the working state of the target equipment according to the predicted temperature value of the target equipment; the temperature prediction model is constructed according to a GRU network, and the temperature prediction model determines a predicted temperature value based on the following modes: determining a plurality of feature units according to sub-images of the infrared thermal image, and fusing the feature units to obtain temperature features of the infrared thermal image; and determining a predicted temperature value of the target equipment according to the temperature characteristics of the infrared thermal image. By adopting the embodiment of the invention, the working state of the target equipment can be flexibly managed according to the predicted temperature value, and the applicability is high.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to a device management method, apparatus, device, and storage medium. Background Technology

[0002] With the continuous development of artificial intelligence, neural network-based recognition technologies (such as palmprint recognition and facial recognition) are widely used in various industries. To address the problems of poor heat dissipation, high temperature, and high power consumption caused by the miniaturization of existing recognition devices, accurate temperature prediction of these devices is necessary. Summary of the Invention

[0003] This application provides a device management method, apparatus, device, and storage medium that can accurately predict the temperature of a target device for device management, and has high applicability.

[0004] On one hand, embodiments of this application provide a device management method, the method comprising: Acquire an infrared thermal image of the target device, and perform image segmentation processing on the infrared thermal image to obtain multiple sub-images; By inputting each sub-image of the infrared thermal image into the temperature prediction model, the predicted temperature value of the infrared thermal image is obtained. The operating status of the target equipment is managed based on the predicted temperature value of the target equipment. The temperature prediction model described above is constructed based on a gated recurrent unit (GRU) network, and the predicted temperature value is determined based on the following method: Based on each sub-image of the infrared thermal image, multiple feature units are determined, and the feature units are fused to obtain the temperature features of the infrared thermal image. Each feature unit is used to characterize the temperature features of one sub-image. Based on the temperature characteristics of the infrared thermal image, the predicted temperature value of the target device is determined.

[0005] On the other hand, embodiments of this application provide a device management apparatus, which includes: The image acquisition module is used to acquire infrared thermal images of the target device and perform image segmentation processing on the infrared thermal images to obtain multiple sub-images; The temperature prediction module is used to input each sub-image of the infrared thermal image into the temperature prediction model to obtain the predicted temperature value of the infrared thermal image. The equipment management module is used to manage the operating status of the target equipment based on the predicted temperature value of the target equipment. The temperature prediction model described above is constructed based on a gated recurrent unit (GRU) network, and the predicted temperature value is determined based on the following method: Based on each sub-image of the infrared thermal image, multiple feature units are determined, and the feature units are fused to obtain the temperature features of the infrared thermal image. Each feature unit is used to characterize the temperature features of one sub-image. Based on the temperature characteristics of the infrared thermal image, the predicted temperature value of the target device is determined.

[0006] On the other hand, embodiments of this application provide an electronic device, including a processor and a memory, which are interconnected; The aforementioned memory is used to store computer programs; The processor described above is used to execute the device management method provided in the embodiments of this application when the computer program described above is invoked.

[0007] On the other hand, embodiments of this application provide a computer-readable storage medium storing a computer program that is executed by a processor to implement the device management method provided in embodiments of this application.

[0008] On the other hand, embodiments of this application provide a computer program product, which includes a computer program that, when executed by a processor, implements the device management method provided in embodiments of this application.

[0009] In this embodiment of the application, by processing the infrared thermal image in blocks, each sub-image can capture local temperature features more precisely, thereby avoiding information loss caused by processing the entire infrared thermal image and improving the accuracy of temperature prediction.

[0010] Furthermore, each feature unit targets a sub-image, increasing sensitivity to local temperature changes and thus improving adaptability and robustness in different environments. By fusing multiple feature units with overall features, both local and global temperature information can be considered, enhancing the expressive power of temperature features in infrared thermal images. A temperature prediction model built on a GRU network effectively handles the correlations between feature units, facilitating the capture of temperature changes in infrared thermal images while offering faster training and inference speeds compared to other deep learning networks. Moreover, the GRU network has fewer parameters, resulting in higher computational efficiency compared to other recurrent neural networks (such as LSTM), making it more suitable for applications with high real-time requirements and enabling flexible management of the target device's operating status based on predicted temperature values. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of an application scenario provided in the embodiments of this application; Figure 2 This is a flowchart illustrating the device management method provided in an embodiment of this application; Figure 3 (a) in this application is a schematic diagram of the device image of the target device provided in an embodiment of the present application; Figure 3 (b) is a schematic diagram of an infrared thermal image provided in an embodiment of this application; Figure 4 This is a schematic diagram of image segmentation according to an embodiment of this application; Figure 5 This is a schematic diagram of the GRU network structure provided in an embodiment of this application; Figure 6 This is a schematic diagram of the camera structure provided in an embodiment of this application; Figure 7 This is a schematic diagram of the network structure of the temperature prediction model provided in the embodiments of this application; Figure 8 This is a schematic diagram of the functional modules provided in the embodiments of this application; Figure 9 This is a data processing diagram provided in an embodiment of this application; Figure 10 This is a schematic diagram of the error curve provided in the embodiments of this application; Figure 11 This is a schematic diagram of the model training process provided in the embodiments of this application; Figure 12 This is a timing diagram of the machine learning workflow implemented in the embodiments of this application; Figure 13 This is a schematic diagram of the model training and integration process provided in the embodiments of this application; Figure 14 This is a schematic diagram of a data processing and analysis architecture implemented in an embodiment of this application; Figure 15 This is a schematic diagram of the model deployment process provided in the embodiments of this application; Figure 16 This is a schematic diagram of the structure of the device management device provided in the embodiments of this application; Figure 17 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0014] The device management method provided in this application can be used to predict the temperature value of a target device, thereby helping to adjust the working mode of the target device in a timely manner based on the temperature value of the target device, saving device power consumption and ensuring stable operation of the device.

[0015] In this application embodiment, the target device can be a device with image capture function and can perform specific object recognition and detection on the captured image to be processed, including but not limited to human body recognition device, face recognition device, palm print recognition device or other object recognition device, etc. The specific device can be determined based on the actual application scenario requirements and is not limited here.

[0016] For example, human body recognition devices (target devices) can enable real-time human body detection and abnormal behavior detection for smart cameras installed in public places such as shopping malls and schools, thereby improving the efficiency of security management.

[0017] For example, a facial recognition device (target device) can be used to analyze facial features for identity verification, recognition, and payment. In this scenario, the facial recognition device (target device) in this application embodiment can be a camera of an access control system, a mobile phone, a laptop, a smart surveillance camera, a human payment terminal, or a camera, etc., without any restrictions.

[0018] For example, palmprint recognition devices (target devices) can perform identity verification, payment, etc., based on human palmprint features. Specifically, palmprint recognition devices (target devices) can be palm-swipe payment devices, palmprint recognition access control systems, etc., without any restrictions.

[0019] In this application embodiment, the target device may also be a traffic camera or other device used in the field of road traffic for vehicle recognition or traffic sign recognition, and there is no limitation.

[0020] See Figure 1 , Figure 1 This is a schematic diagram of an application scenario provided in the embodiments of this application.

[0021] like Figure 1As shown, assuming the target device 11 is a palm payment device, the target device 11 continuously takes pictures during the palm payment process of the target object 12 so as to realize palm recognition and payment function when the palm is close to the target device 11.

[0022] The embodiments of this application can be executed by the target device 11, or by a server associated with the target device or a terminal device with image processing capabilities. The specific execution method can be determined based on the actual application scenario requirements and is not limited here.

[0023] Among them, the server can be an independent physical server, such as a video server or a streaming media server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0024] The terminal device can be a smartphone, tablet, laptop, desktop computer (including monitor), smartwatch, vehicle terminal, aircraft, smart home appliance (such as smart TV), or wearable device with image processing capabilities.

[0025] See Figure 2 , Figure 2 This is a flowchart illustrating the device management method provided in an embodiment of this application. Figure 2 As shown, the device management method provided in this application embodiment may include the following steps: S21, acquire the infrared thermal image of the target device, and perform image block processing on the infrared thermal image to obtain multiple sub-images.

[0026] In some feasible implementations, infrared thermal images of the target device can be acquired using an infrared thermal imager. An infrared thermal imager is a device capable of measuring and displaying the temperature distribution on the surface of an object. It utilizes the principle of infrared radiation to convert the infrared radiation energy on the object's surface into a visible image, thereby intuitively displaying the object's heat distribution.

[0027] The infrared thermal image of the target device can be an infrared thermal image of the target device's surface, or an infrared thermal image of the target device's surface at different angles determined by an infrared thermal imager. The specific determination can be based on the actual application scenario requirements and is not limited here.

[0028] Figure 3 (a) is a schematic diagram of the device image of the target device provided in an embodiment of this application. Figure 3 The target device shown in (a) is a palm payment device. Infrared thermal imagers can be used to obtain... Figure 3 Infrared thermal image of the target device shown in (b) of the image.

[0029] In some feasible implementations, after acquiring an infrared thermal image, the infrared thermal image can be pre-processed into image blocks to obtain multiple sub-images.

[0030] Specifically, the infrared thermal image can be divided into multiple sub-images of the same size based on a preset block window.

[0031] For example, see Figure 4 , Figure 4 This is a schematic diagram of image segmentation according to an embodiment of this application. For example... Figure 4 As shown, assuming the infrared thermal image of the target device has an image size of 9×9 and a preset block window size of 3×3, with a block step size of 3, the infrared thermal image can be divided into 9 non-overlapping sub-images with an image size of 3×3.

[0032] Optionally, when processing the infrared thermal image into blocks, the infrared thermal image can be pre-filtered and denoised to obtain a denoised image, and then the denoised image can be divided into multiple sub-images according to a preset block window.

[0033] Alternatively, when processing infrared thermal images in blocks, the infrared thermal images can be pre-divided into multiple initial images according to a preset block window. Then, each initial image can be filtered and denoised to obtain a denoised image. Thus, the denoised image corresponding to each initial image can be determined as a sub-image of the infrared thermal image.

[0034] By filtering and denoising the image, noise such as impulse noise and additive Gaussian noise can be effectively suppressed, improving the clarity of the infrared thermal image while better preserving the edge information of the infrared thermal image.

[0035] When performing filtering and noise reduction on infrared thermal images, methods such as median filtering, mean filtering, Gaussian filtering, and bilateral filtering can be used. The specific method can be determined based on the actual application scenario requirements and is not limited here.

[0036] As an example, when performing image block processing on an infrared thermal image to obtain multiple sub-images, the neighborhood window of each pixel in the infrared thermal image can be determined according to a preset neighborhood window.

[0037] When any pixel is located at the edge of the infrared thermal image, reflection filling or mirror filling can be used to fill in the missing part of the pixel.

[0038] Furthermore, for each pixel, the median value of the original pixel values ​​of all other pixels within the neighborhood window of that pixel can be determined, and the original pixel value of that pixel can be replaced with the median value.

[0039] After replacing each pixel with the median value corresponding to the neighborhood window, a denoised image can be obtained. At this time, the denoised image can be divided into blocks by a preset block window to obtain sub-images of the infrared thermal image.

[0040] Alternatively, when performing image segmentation processing on an infrared thermal image to obtain multiple sub-images, the infrared thermal image can be separated into multiple initial images according to a preset segmentation window.

[0041] Furthermore, for each initial image, a neighborhood window for each pixel in the initial image can be determined based on a preset neighborhood window. Similarly, when any pixel is located at the edge of the initial image, the missing part of the pixel can be filled using reflection filling or mirror filling.

[0042] Furthermore, for each pixel in each initial image, the median value of the original pixel values ​​of all other pixels within the neighborhood window of that pixel can be determined, and the original pixel value of that pixel can be replaced with the median value. After replacing each pixel in the initial image with the median value corresponding to the neighborhood window, a denoised image is obtained, which can then be identified as a sub-image of the infrared thermal image.

[0043] It should be noted that in the embodiments of this application, the multiple sub-images corresponding to the infrared thermal image may not overlap, or some sub-images may overlap. The specific overlap can be determined based on the block step size, and there is no limitation here.

[0044] In this embodiment, noise can be effectively removed and image quality improved by using neighborhood windows and median replacement, providing a better foundation for subsequent feature extraction. Furthermore, the block processing after denoising allows for more detailed extraction of local temperature features, enhancing overall prediction performance.

[0045] S22, input each sub-image of the infrared thermal image into the temperature prediction model to obtain the predicted temperature value of the target device.

[0046] In some feasible implementations, after determining the individual sub-images of the infrared thermal image, the individual sub-images can be input into a pre-trained temperature prediction model to obtain the predicted temperature value of the target device.

[0047] The predicted temperature value of the target device can be the average temperature or the highest temperature of the target device surface, which can be determined based on the actual application scenario requirements and is not limited here.

[0048] The temperature prediction model provided in this application embodiment can be constructed based on a Gated Recurrent Unit (GRU) network. The GRU network is a variant of a recurrent neural network designed to address the vanishing gradient problem that occurs in standard recurrent neural networks when processing long sequences of data. By introducing a gating mechanism, GRU can better handle long-term dependencies, has a simpler structure, and is more computationally efficient.

[0049] The temperature prediction model in this embodiment can determine the predicted temperature value of the target device in the following ways: The infrared thermal image temperature features are obtained by using a feature extraction network to determine multiple feature units based on each sub-image of the infrared thermal image and by fusing the feature units. The predicted temperature value of the target device is determined by a temperature prediction network based on the temperature characteristics of infrared thermal images.

[0050] In the temperature prediction model, the feature extraction network and the temperature prediction network can be constructed based on the GRU network.

[0051] In some feasible implementations, the feature extraction network includes a GRU network. After the feature extraction network acquires each sub-image of the infrared thermal image, it can use each sub-image as input to the GRU network and then output the feature units of each sub-image.

[0052] Each feature unit of a sub-image is used to characterize the temperature features of the corresponding sub-image.

[0053] See Figure 5 , Figure 5 This is a schematic diagram of the GRU network structure provided in an embodiment of this application. Figure 5 As shown, each sub-image of the infrared thermal image serves as the network input, and the input (X) at each time step... t A is a sub-image of an infrared thermal image. This sub-image is typically converted into a feature vector (Y). t This vector contains the temperature information of the sub-image.

[0054] Each GRU unit receives input (X) at the current time step. t ) and the hidden state of the previous time step (H) t-1 It takes information as input. It uses three "gates" to control the flow of information: the Update Gate, the Reset Gate, and the Candidate Gate. These gates are computed using the sigmoid activation function and matrix multiplication to determine which information should be retained, which should be forgotten, and how new information should be integrated into the hidden state.

[0055] Based on this, after inputting each sub-image into the GRU network, each sub-image X can be obtained. t The corresponding temperature characteristic Y t .

[0056] When fusing various feature units to obtain the temperature features of an infrared thermal image, the feature units can be stitched together in the order of their position information corresponding to the infrared thermal image to obtain the temperature features of the infrared thermal image.

[0057] Optionally, when fusing the feature units to obtain the temperature features of the infrared thermal image, the vector elements of each feature unit can be added together to obtain the temperature features of the infrared thermal image.

[0058] Optionally, when fusing the feature units to obtain the temperature features of the infrared thermal image, the weights corresponding to each feature unit can be determined, and then the feature units can be weighted and summed to obtain the temperature features of the infrared thermal image.

[0059] The weight of each feature unit can be determined based on the proportion of the target device included in the corresponding sub-image. The larger the weight, the more of the target device is included in the corresponding sub-image.

[0060] Optionally, when fusing the feature units to obtain the temperature features of the infrared thermal image, the vector elements of each feature unit can be multiplied together to obtain the temperature features of the infrared thermal image.

[0061] Optionally, when fusing the feature units to obtain the temperature features of the infrared thermal image, the temperature information of at least one hardware component of the target device can be determined.

[0062] By fusing feature units in the above manner, the spatial relationship between sub-images can be effectively preserved, the accuracy of temperature features in infrared thermal images can be improved, and the structured processing of temperature features can be made simpler and more effective.

[0063] The hardware components of the target device include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), double data rate synchronous dynamic random access memory (DDR), an embedded multi media card (EMMC), and input / output (IO) components. The specific components can be determined based on the actual application scenario requirements and are not limited here.

[0064] The hardware components of the target device can be obtained by reading the relevant paths in the target device's system files, or by using high-precision temperature sensors installed in key components such as the GPU and CPU of the target device; there are no restrictions on this.

[0065] In addition to the aforementioned GPU and other components, the hardware components of the target device may also include an RGB camera, an infrared (IR) camera, an RGB aperture, and an infrared light-emitting diode (LED), etc. See also... Figure 6 , Figure 6 This is a schematic diagram of the camera structure provided in an embodiment of this application. Figure 6 The camera structure shown can be used in target devices such as palm payment devices and facial recognition devices. The camera structure may include an RGB camera, an IR camera, an RGB aperture, an IR emission polarization region, an IR receiving polarization region, and two infrared LEDs.

[0066] Infrared light-emitting diodes (IR LEDs) are diodes that emit infrared light and are typically used in applications requiring night vision or low-light operation, such as cameras and infrared remote controls. The main function of IR LEDs is to provide an infrared light source in low-light or completely dark environments, enabling cameras to capture and record clear images.

[0067] RGB cameras use three standard channels—red, green, and blue—to capture color information within the visible light spectrum. They generate color images by varying these three color channels and superimposing them onto each other, thus producing a wide variety of colors.

[0068] Among them, IR cameras use infrared sensors to capture radiation within the infrared spectrum. These cameras are able to sense infrared radiation emitted or reflected by objects, rather than visible light.

[0069] RGB light guide rings typically refer to ring-shaped light sources used for lighting or decoration. They emit red, green, and blue light, and produce various colors of light by controlling the mixing of different colors. RGB light guide rings are commonly found in display devices, lighting equipment, and electronic products to enhance their visual appeal.

[0070] In this embodiment of the application, the supplementary light source can be an RGB light guide ring, that is... Figure 6 The color temperature of the RGB light guide ring is set to 6500K.

[0071] In this context, the IR emission polarization region refers to the area where the infrared transmitter emits infrared signals, while the IR reception polarization region refers to the area where the infrared receiver receives infrared signals. By using polarization technology, it is possible to ensure that the infrared signal has a certain directionality and anti-interference capability during transmission and reception.

[0072] Since the hardware components of the target device are usually measured in millidegrees Celsius, while the temperature value of the target device is usually evaluated in degrees Celsius, the temperature data of the hardware components of the target device can be converted into degrees Celsius to obtain temperature information.

[0073] After acquiring the temperature information of at least one hardware component, for each feature unit, attention features can be determined based on the temperature information of each hardware component, and then a first fusion feature can be determined based on the attention features and the feature unit. The first fusion feature corresponding to each feature unit is a related feature that injects the temperature information of each hardware component and is used to characterize the temperature features of the corresponding sub-image.

[0074] Based on this, the first fusion feature corresponding to each feature unit can be fused to obtain the temperature feature of the infrared thermal image.

[0075] When fusing the first fusion features corresponding to each feature unit, the first fusion features can be concatenated according to the positional information of the corresponding sub-images in the infrared thermal image to obtain the temperature features of the infrared thermal image. Alternatively, the vector elements of each first fusion feature can be added together to obtain the temperature features of the infrared thermal image. Alternatively, the weights of each first fusion feature can be determined, and then the first fusion features can be weighted and summed to obtain the temperature features of the infrared thermal image. The weights of each first fusion feature can be determined based on the proportion of the target device portion included in the corresponding sub-image; a larger weight indicates that the corresponding sub-image includes a larger portion of the target device. Alternatively, the vector elements of each first fusion feature can be multiplied together to obtain the temperature features of the infrared thermal image.

[0076] For each feature unit, when determining the attention feature based on the temperature information of the feature unit and each hardware component, and then determining the first fusion feature based on the attention feature and the feature unit, the weights applied to the feature unit by each hardware component can be calculated separately, and the weights corresponding to each hardware component can be normalized to obtain the target weights. Then, the feature unit is multiplied by the sum of the target weights of each hardware component to obtain the temperature feature of the feature unit.

[0077] The weights corresponding to each hardware component can be regarded as attention features.

[0078] In this embodiment, by introducing temperature information and attention features from hardware components, the various feature units can be more accurately fused, improving the expressive power of temperature features. Furthermore, dynamic adjustments based on real-time temperature information from the hardware components help capture environmental changes and improve the real-time performance of temperature features.

[0079] In some feasible implementations, the temperature prediction network includes a GRU network. After the temperature prediction network obtains the temperature features, it can process the temperature features of the infrared thermal image through a convolutional network to obtain convolutional features.

[0080] Furthermore, the temperature features of the infrared thermal image are processed by the GRU network to obtain intermediate features, which are the output of the GRU network. Then, the intermediate features are processed by the attention fusion layer based on the self-attention mechanism to obtain attention fusion features.

[0081] Based on this, the temperature prediction network can fuse the convolutional network and attention fusion features to obtain a second fusion feature, and determine the predicted temperature value of the target device based on the second fusion feature.

[0082] Specifically, the temperature prediction network can process the temperature features of infrared thermal images through at least one first network and a flattening layer. Each first network includes a convolutional layer and a pooling layer. The first networks are connected in sequence and finally connected to the flattening layer.

[0083] The second fused feature can be obtained by fusing convolutional features and attention fusion features through feature concatenation.

[0084] Specifically, when determining the predicted temperature value of the target device based on the second fusion feature, the second fusion feature can be processed through a fully connected layer to finally output the predicted temperature value of the target device.

[0085] As an example, see Figure 7 , Figure 7 This is a schematic diagram of the network structure of the temperature prediction model provided in an embodiment of this application. For example... Figure 7 As shown, the temperature prediction model includes a feature extraction network. After determining the feature units of each sub-image through a GRU network, the feature extraction network can determine attention features based on the temperature information of at least one hardware component of the target device. Then, based on the attention features and each feature unit, it determines the corresponding first fusion feature. Finally, the feature extraction network can stitch together the first fusion features corresponding to each feature unit to obtain the temperature features of the determined infrared thermal image.

[0086] Furthermore, the temperature prediction model includes a temperature prediction network. A first convolutional layer performs a convolution operation on the temperature features to extract local features. A first pooling layer downsamples the convolutional features to reduce the feature map size. A second convolutional layer performs another convolution operation to further extract features. A second pooling layer performs another pooling operation, and a flattening layer flattens the output of the second pooling layer into a one-dimensional vector.

[0087] On the other hand, the temperature prediction network processes the input temperature features through a GRU network to obtain intermediate features, and then uses an attention layer to obtain attention fusion features through a self-attention mechanism. The attention fusion features and the output of the flattening layer are concatenated to obtain the second fusion feature. Finally, further feature extraction is performed through a fully connected layer, and the extracted features are processed through a dropout layer, randomly discarding a certain proportion of neurons to prevent overfitting. The predicted temperature value of the target device is then determined based on the output of the dropout layer.

[0088] In this embodiment, by processing the infrared thermal image in blocks, each sub-image can capture local temperature features more precisely, thereby avoiding information loss caused by processing the entire infrared thermal image and improving the accuracy of temperature prediction. Through a feature extraction network, complex temperature features in the infrared thermal image can be automatically extracted, rather than relying on manually designed features. This allows for better adaptation to different infrared thermal image representations, improving the accuracy and applicability of the equipment management method.

[0089] Furthermore, each feature unit targets a sub-image, increasing sensitivity to local temperature changes and thus improving adaptability and robustness in different environments. By fusing multiple feature units with overall features, both local and global temperature information can be considered, enhancing the expressive power of temperature features in infrared thermal images. A temperature prediction model built on a GRU network effectively handles the correlations between feature units, facilitating the capture of temperature changes in infrared thermal images while offering faster training and inference speeds compared to other deep learning networks. Moreover, the GRU network has fewer parameters, making it more computationally efficient than other recurrent neural networks (such as LSTM), and more suitable for applications with high real-time requirements.

[0090] S23, manage the operating status of the target equipment based on the predicted temperature value of the target equipment.

[0091] In some feasible implementations, the target device can be used to identify target objects, such as palmprint recognition (the target object is a palm), face recognition (the target object is a face), human body recognition (the target object is a human body), and object recognition (the target object is a specified object).

[0092] In this case, after obtaining the predicted temperature value of the target device based on the device management method provided in the embodiments of this application, the working status of the target device can be managed according to the predicted temperature value of the target device, so that the target device can work with optimal performance while avoiding excessive temperature.

[0093] Specifically, if the predicted temperature value of the target device is higher than the preset temperature value, and there is no target object approaching the target device within the target device's recognition distance range, the target device can be controlled to operate in a first operating mode. If there is a target object approaching the target device within the target device's recognition distance range, the target device is controlled to operate in a second operating mode.

[0094] If the predicted temperature value of the target device is lower than the preset temperature value, the target device is controlled to operate in the second working mode.

[0095] In the first operating mode, the power consumption of the target device is less than that in the second operating mode.

[0096] In other words, when the predicted temperature value of the target device is higher than the preset temperature value, it means that the temperature of the target device is high. If there is no target object approaching the target device, that is, if the target device does not need to identify any target object, the target device can be controlled to work in the first working mode, that is, to work in the low-energy mode.

[0097] When the predicted temperature value of the target device is higher than the preset temperature value, but there is a target object approaching the target device within the identification distance range, it means that the target device needs to identify the target object. At this time, the target device needs to work in the first working mode, that is, it cannot work in the normal power consumption mode.

[0098] When the predicted temperature value of the target device is lower than the preset temperature value, it means that the temperature of the target device is within the normal range and there is no abnormality such as power overload. At this time, the target device can be controlled to work in the second working mode, that is, to maintain normal energy consumption mode.

[0099] The control of the target device to operate in the first working mode can be achieved by reducing the target device's operating frequency, controlling the target device to enter a power-saving mode, controlling the target device to enter a sleep mode, turning off the target device's noise reduction function, turning off the target device's image signal processing (ISP) function, reducing the target device's screen brightness, or reducing the target device's power amplifier (PA) level, etc. The specific method can be determined based on the actual application scenario requirements and is not limited here.

[0100] In this embodiment of the application, by dynamically adjusting the working mode of the target device based on the predicted temperature value and the proximity of the target object, energy consumption can be effectively saved and the working efficiency of the device can be improved.

[0101] In some feasible implementations, when determining whether there is a target object approaching the target device within the recognition distance range, the first data sequence can be determined if the target object appears within the recognition distance range of the target object.

[0102] The first data sequence includes distance information between the target device and the target object acquired at preset time intervals after the target device appears within the recognition distance range. This distance information can be determined by a distance sensor deployed on the target device.

[0103] Furthermore, the first data sequence can be input into a pre-trained behavior prediction model to obtain the behavior prediction result of the target object.

[0104] The first time interval is the time interval corresponding to the first data sequence.

[0105] Based on this, it can be determined whether the target object is approaching the target device based on the predicted behavior of the target object, thereby determining whether the target device needs to identify the target object.

[0106] For example, the target device may include Figure 8 The functional modules shown are as follows: Figure 8 This is a schematic diagram of the functional modules provided in an embodiment of this application. The distance sensor measurement module is used to determine whether a target object exists within the identification distance range, and, if a target object exists, to acquire distance information (a first data sequence) between itself and the target object at preset time intervals.

[0107] The preprocessing module is used to perform preprocessing operations such as filtering and normalization on the distance information to facilitate analysis by the behavior prediction module.

[0108] The behavior prediction module is used to determine the behavior prediction results of the target object based on the data from the preprocessed model, such as the approach speed and dwell time within the corresponding time interval.

[0109] The adjustment module is used to manage the working status of the target device based on the behavior prediction results.

[0110] In this embodiment, the behavior prediction model can predict the future actions of a target object in advance, thereby enabling the management of the target device's operating state. This helps reduce the device's power consumption and improves the flexibility of its operating state. Furthermore, the data sequence allows for more accurate analysis of the target object's behavioral trends, enhancing the accuracy and reliability of the prediction.

[0111] In this application, the behavior prediction model can be built based on a GRU network or an LSTM network, and the specific model can be determined based on the actual application scenario requirements, without any restrictions.

[0112] In some feasible implementations, the behavior prediction result of the target object may include at least one of the following: The behavioral state of the target object, including whether it is approaching or moving away from the target device; The approach speed of the target object; The speed at which the target object moves away; The time spent by the target audience.

[0113] The dwell time of the target object can be the dwell time of the target object within the distance recognition range of the target device.

[0114] Based on this, when managing the working status of the target device according to the behavior prediction results of the target object, it can be determined whether the behavior prediction results of the target object meet the preset conditions.

[0115] Specifically, the predicted behavior of the target object meets preset conditions, which may include any one or more of the following: The target object is approaching the target device; The target object's approach speed is greater than the preset approach speed; The target object's dwell time is longer than the preset dwell time.

[0116] In this context, the approach of a target object to the target device indicates that the target device is about to identify the target object. If the approach speed of the target object exceeds a preset approach speed and / or the dwell time of the target object exceeds a preset dwell time, it indicates a very high probability that the target object will need to use the target device, and consequently, a very high probability that the target device will need to identify the target object.

[0117] In this scenario, when the predicted behavior of the target object meets the preset conditions and the target device is in sleep mode, the target device can be controlled to enter wake-up mode for target object identification. When the predicted behavior of the target object meets the preset conditions and the target device is in wake-up mode, it indicates that the target device is already operational, and no adjustment to its operational status is necessary.

[0118] Specifically, the predicted behavior of the target object does not meet the preset conditions, which may include any one or more of the following: The target object is far away from the target device; The target object's moving away speed is greater than the preset moving away speed; The target object's dwell time is less than or equal to the preset dwell time.

[0119] If the target object moves away from the target device, and / or the target object's speed is greater than the preset moving away speed, and / or the target object's dwell time is less than the preset dwell time, it indicates that the target object does not have any need to use the target object, and the target device does not need to identify the target object.

[0120] In this scenario, if the predicted behavior of the target object does not meet the preset conditions and the target device is in a wake-up state, the target device is controlled to enter a sleep state to reduce power consumption. When the predicted behavior of the target object does not meet the preset conditions and the target device is in a sleep state, it indicates that the target device is already operating in a low-power mode, therefore no adjustment to the target device's operating state is necessary.

[0121] In some feasible implementations, when managing the working state of the target device based on the behavior prediction results of the target object, a working frequency that matches the behavior prediction results can also be determined, thereby controlling the target device to work at the matching working frequency.

[0122] The operating frequency in this application embodiment includes, but is not limited to, processor frequency, screen refresh rate, data transmission rate, sampling rate (such as camera frame rate), etc., which can be determined based on the actual application scenario requirements and are not limited here.

[0123] Different behavior prediction results correspond to different operating frequency configuration information. After determining the behavior prediction result of the target object, the matching frequency configuration information can be directly determined, and then the target device can be controlled to work at the corresponding operating frequency based on the determined frequency configuration information.

[0124] Optionally, when the behavior prediction result includes the behavior state, the operating frequency of the target device when the target object is close to the target device is greater than the operating frequency when the target object is far away from the target device. That is, when the target object is close to the target device, the target device needs to operate at a high operating frequency to accurately identify the target object, and when the target object is far away from the target device, the target device needs to operate at a low operating frequency to reduce device power consumption and lower device temperature.

[0125] Optionally, when the behavior prediction result includes approach speed, the operating frequency of the target device is positively correlated with the approach speed. That is, when the target object approaches the target device at different speeds, the target device can operate at different frequencies, and the greater the approach speed of the target object, the higher the operating frequency the target device needs to operate at in order to improve the recognition speed and accuracy.

[0126] The correspondence between approach speed and operating frequency can be pre-configured. For example, a mapping relationship between different approach speeds and different operating frequencies can be pre-established. After determining the approach speed of the target object, the matching frequency configuration information can be directly determined, and the target device can be controlled to work at the corresponding operating frequency, thereby improving work efficiency.

[0127] Optionally, when the behavior prediction result includes dwell time, the operating frequency of the target device is negatively correlated with the dwell time. That is, depending on the dwell time of the target object, the target device needs to complete the identification of the target object within different time periods. Therefore, when the dwell time of the target object is short, the target device needs to operate at a higher frequency to save time. When the dwell time of the target object is long, the target device can operate at a lower frequency to reduce device power consumption and lower device temperature while completing the identification process.

[0128] The correspondence between dwell time and working frequency can be pre-configured. For example, a mapping relationship between different dwell times and different working frequencies can be pre-established. After determining the dwell time of the target object, the frequency configuration information that matches it can be directly determined, and the target device can be controlled to work at the corresponding working frequency, thereby improving work efficiency.

[0129] Optionally, when the behavior prediction result includes the distance moving away from the target device, the operating frequency of the target device is negatively correlated with the distance moving away from the target device. That is, when the target object moves away from the target device at different speeds, the target device can operate at different frequencies. Furthermore, the greater the distance moving away from the target object, the greater the probability that the target object will no longer use the target device. In this case, the target object can operate at a lower frequency to reduce device power consumption.

[0130] The relationship between the moving-away speed and the operating frequency can be pre-configured. For example, a mapping relationship between different moving-away speeds and different operating frequencies can be pre-established. After determining the moving-away speed of the target object, the matching frequency configuration information can be directly determined, and the target device can be controlled to work at the corresponding operating frequency, thereby improving work efficiency.

[0131] Optionally, in the embodiments of this application, if the operating frequency is the frame rate of the target device's camera, the frame rate of the camera can be controlled by a frame rate sensor.

[0132] The frame rate of the camera includes, but is not limited to, one or more of the frame rates of the RGB sensor and the IR sensor.

[0133] In this embodiment of the application, the working state of the target device is managed according to the behavior prediction results of the target object. This can effectively adjust the working state of the target device in a timely manner according to the future behavior trend of the target object, such as adjusting the working frequency of the target device. This helps to ensure the recognition accuracy of the target device and also helps to avoid the target device consuming unnecessary power.

[0134] In some feasible implementations, the temperature prediction model in the embodiments of this application can be obtained by training a training sample set and a validation sample set.

[0135] The training sample set includes multiple first image sets, each first image set includes multiple sub-images of a first infrared thermal image, and each first image set corresponds to a different first infrared thermal image.

[0136] The verification sample set includes multiple second image sets, each second image set includes multiple sub-images of a second infrared thermal image, and each second image set corresponds to a different second infrared thermal image.

[0137] Each first infrared thermal image and each second infrared thermal image are infrared thermal images of the target device determined at different times and under different working environments.

[0138] In this context, the second infrared thermal image corresponding to any second image set is different from the infrared thermal images corresponding to each first image set.

[0139] After determining the training and validation sample sets, the initial model can be trained iteratively a predetermined number of times using the training sample set to obtain the temperature prediction model. The initial model includes the aforementioned feature extraction network and temperature prediction network.

[0140] In each iteration, each sub-image from each first image set can be input into the initial model obtained from the previous iteration to obtain the predicted temperature value of the target device corresponding to each first image set. For the first iteration, each sub-image from each first image set can be input into the initial model to obtain the predicted temperature value of each first image combined with the corresponding target device.

[0141] Furthermore, the first training loss can be determined based on the actual temperature value and the predicted temperature value of the target device corresponding to each first infrared thermal image.

[0142] The actual temperature value of the target device corresponding to each first infrared thermal image can be determined by measuring it using a temperature sensor or other temperature detection device when determining the training sample set.

[0143] The first training loss can be determined using functions such as Mean Squared Error (MSE) and Mean Absolute Error (MAE). The specific loss can be determined based on the actual application scenario requirements and is not limited here.

[0144] Furthermore, after each iteration, each set of second images is input into the initial model obtained in this iteration to obtain the predicted temperature value of the target device corresponding to each second infrared thermal image. Then, based on the predicted temperature value and the actual temperature value of the target device corresponding to each second infrared thermal image, the first verification loss is determined.

[0145] The initial model determines the predicted temperature value of the target device for each first image set and the predicted temperature value of the target device for each second image set in the same way as the temperature prediction network determines the predicted temperature value, and will not be described again here.

[0146] The actual temperature value of the target device corresponding to each second infrared thermal image can be determined by measuring it using a temperature sensor or other temperature detection device when determining the training sample set.

[0147] Optionally, when the first infrared thermal images corresponding to the training sample set are infrared thermal images of the target device operating at different times, the actual temperature value of the target device corresponding to each first infrared thermal image can be the average of the original temperature values ​​of multiple target devices at that moment. Similarly, when the second infrared thermal images corresponding to the validation sample set are infrared thermal images of the target device operating at different times, the actual temperature value of the target device corresponding to each second infrared thermal image can be the average of the original temperature values ​​of multiple target devices at that moment. By using the average temperature value in training or validation, the poor training effect caused by the existence of abnormal temperature values ​​under extreme conditions can be avoided, which is beneficial to improving the model training effect.

[0148] For example, see Figure 9 , Figure 9 This is a schematic diagram of data processing provided in an embodiment of this application. For example... Figure 9 As shown, taking the training sample set as an example, after obtaining the original temperature value of the target device at different times, the original temperature value in each time period can be averaged based on the Piecewise Aggregate Approximation (PPA) method to obtain the average value of the original temperature value of the target device in each time period, and then use it as the actual temperature value for training.

[0149] The determination of the second validation loss and the first training loss can be made using functions such as Mean Squared Error (MSE) and Mean Absolute Error (MAE). The specific determination can be based on the actual application scenario requirements and is not limited here.

[0150] Based on this, after completing a preset number of iterations of training, the final temperature prediction model can be determined from the initial model obtained in each iteration process based on the first loss set and the second loss set.

[0151] The first loss set includes the first training loss obtained in each iteration, and the second loss set includes the first validation loss obtained after each iteration.

[0152] For example, the initial model obtained by the iterative process that simultaneously makes each of the first training losses in the first loss set converge and each of the first validation losses in the second loss set converge can be determined as the final temperature prediction model.

[0153] In this embodiment, the initial model is iteratively trained using detailed training and validation sample sets, which systematically optimizes the initial model and improves its generalization ability. Furthermore, the initial model can be dynamically adjusted based on the difference between the actual and predicted temperature values ​​during each iteration, thereby gradually optimizing the initial model and improving training effectiveness.

[0154] In some feasible implementations, when determining the temperature prediction model from the initial model obtained in each iteration process based on the first loss set and the second loss set, the first verification loss in the second loss set can be smoothed in advance to obtain the verification error curve, and it can be determined whether there are outliers in the verification error curve that meet the preset conditions.

[0155] Among them, outliers that meet the preset conditions can be error values ​​that suddenly increase the verification error. For example, if the error value corresponding to a certain iteration process in the verification error curve is higher than the error value corresponding to the previous iteration process in the verification error curve, and the difference between the two is greater than the preset difference, the error value corresponding to the verification error curve of that iteration process can be identified as an outlier.

[0156] When the verification error curve contains outliers that meet the preset submission criteria, it indicates that the initial model starts fitting from the iteration process corresponding to the outlier. At this time, the initial model obtained from the iteration process corresponding to the outlier can be determined as the final temperature prediction model.

[0157] When the validation error curve does not contain outliers that meet the preset conditions, it indicates that the initial model has not reached the overfitting state. At this time, the first training loss in the first loss set can be further smoothed to obtain the training error curve, and it can be determined whether the training error curve and the validation error curve intersect.

[0158] When the training error curve and the validation error curve intersect, it indicates that the initial model has reached a performance balance in the iteration process corresponding to the intersection point. At this point, the initial model obtained from the iteration process corresponding to the intersection point can be determined as the final temperature prediction model.

[0159] When the training error curve and the validation error curve do not intersect, the temperature prediction model is determined from the initial model obtained in each iteration based on the training error curve. For example, the initial model obtained in the iteration process when the first training loss is minimized is determined as the final temperature prediction model, or the initial model obtained in the iteration process when the first training loss reaches convergence is determined as the final temperature prediction model.

[0160] In this embodiment, smoothing the validation loss effectively removes the interference of outliers, ensuring the stability and reliability of the initial model. Using features such as crossover points or minimum values ​​for model selection helps find the optimal model and improves prediction performance.

[0161] Optionally, if the training error curve and the validation error curve do not intersect, when the training error curve reaches the convergence condition, the initial model obtained in the Mth iteration and the initial model obtained in the Nth iteration when the training error curve reaches convergence are determined as the final temperature prediction model.

[0162] The Mth iteration is the iteration process corresponding to the minimum value in the training error curve.

[0163] For example, when M is greater than N, the initial model obtained in the Nth iteration can be determined as the final temperature prediction model; when N is greater than M, the initial model obtained in the Mth iteration can be determined as the final temperature prediction model.

[0164] If the training error curve and the validation error curve do not intersect, the initial model obtained in the Mth iteration can be directly determined as the final temperature prediction model, and the performance of the initial model obtained in the Mth iteration reaches its optimal level.

[0165] For example, see Figure 10 , Figure 10This is a schematic diagram of error curves provided in an embodiment of this application. The validation error curve is obtained by smoothing the first validation loss determined by the mean absolute error (MAE) function, and the training error curve is obtained by smoothing the first training loss determined by the mean absolute error (MAE) function. Figure 10 In the example, assuming the initial model requires 200 iterations of training, then... Figure 10 The validation error curve does not contain any outliers that meet the preset conditions, the training error curve does not contain any outliers that meet the preset conditions, the validation error curve and the training error curve do not intersect, and the training error curve has not converged.

[0166] In this case, the initial model obtained from the iterative process corresponding to the minimum value in the training error curve can be determined as the final temperature prediction model.

[0167] Based on the above training method, the initial model can learn fully from the training data (training dataset) during the training process, while avoiding overfitting, thus maintaining good performance on unseen data (validation sample set).

[0168] The following is combined with Figure 11 The model training process provided in the embodiments of this application will be further explained. Figure 11 This is a schematic diagram of the model training process provided in an embodiment of this application. For example... Figure 11 As shown, the model training process in this embodiment mainly includes the following steps: The data collection phase is used to determine the training sample set and validation sample set for participating in model training.

[0169] The data annotation stage is used to mark the association between each sub-image in each first image set and the corresponding first infrared thermal image, as well as the association between each sub-image in each second image set and the corresponding second infrared thermal image, and to mark the actual temperature value of the target device corresponding to each first infrared thermal image and the second infrared thermal image.

[0170] The data preprocessing stage cleanses the first and second infrared thermal images by finding duplicates, missing values, and outliers to remove invalid data. It also filters and reduces noise in both the first and second infrared thermal images.

[0171] The model building phase is used to construct an initial model, including a feature extraction network and a temperature prediction network.

[0172] The weight initialization step is used to initialize the weights of the initial model to prevent gradient vanishing or exploding, and to help the model find the optimal solution faster, thereby accelerating convergence and improving training efficiency. Inappropriate weight initialization can lead to gradient vanishing or exploding, causing model training failure. Proper weight initialization helps gradients propagate normally, ensuring the model can be trained effectively.

[0173] The weights can be initialized using methods such as zero initialization, random initialization, Xavier / Glorot initialization, or He initialization, without any restrictions.

[0174] The model training stage is used to iteratively train the initial model according to the model training method provided above, so as to obtain the temperature prediction model.

[0175] The model evaluation stage is used to measure model performance, understand how the model performs on the training and validation sets, and feed the evaluation results back into the model training process to help adjust hyperparameters and architecture to improve model accuracy and robustness, so as to ensure that the model can make accurate predictions.

[0176] The hyperparameter tuning stage is used to determine the optimal combination of parameters that enables the initial model to perform best on the validation data, thereby improving the model's performance and effectively preventing overfitting or underfitting.

[0177] In the predictive application stage, an arbitrary infrared thermal image of the target device is determined, and the predicted temperature value of the target device is predicted based on this image.

[0178] In the model training process, multiple temperature prediction models can be trained using training sample sets and validation sample sets respectively. Then, the parameters of the trained multiple temperature predictions are integrated to obtain the final temperature prediction model.

[0179] See Figure 12 , Figure 12 This is a timing diagram illustrating the machine learning workflow implemented in an embodiment of this application. For example... Figure 12As shown, the Monitoring System is responsible for collecting historical data to provide the data foundation for machine learning. This data may come from various sources such as servers, applications, and devices. Since the raw data may contain noise, missing values, or inconsistent formats, the Data Preprocessing stage cleans, transforms, and formats this data to build training and validation sample sets. In the Feature Engineering stage, useful features for model training are extracted from the preprocessed training and validation sample sets, helping the model make accurate predictions. The Deep Learning Model uses the extracted features and performance metrics data to train the temperature prediction model and determine the predicted temperature values ​​in the process. Cloud Infrastructure: The trained model is deployed to the cloud infrastructure for real-time prediction in a real-world environment. The cloud infrastructure provides elastic scaling and fault tolerance, ensuring high availability and stability of the model. After deployment, the model compares the actual temperature values ​​with the predicted values ​​and generates feedback. This feedback is used to fine-tune the model to reduce resource usage and optimize resource allocation. Predictive autoscaling automatically adjusts the allocation of computing resources based on model-predicted temperature values. For example, if predicted temperature values ​​indicate a continued rise, potentially increasing data processing and analysis tasks, the system can automatically add more computing nodes or upgrade the computing power of existing nodes to meet the demand. This autoscaling helps maintain system stability and performance while optimizing resource utilization.

[0180] Based on this, the detection system can perform model fine-tuning feedback according to the actual temperature value and the predicted temperature value, thereby completing the model training process.

[0181] See Figure 13 , Figure 13 This is a schematic diagram of the model training and integration process provided in the embodiments of this application. After determining the training sample set (Tr) and the validation sample set (Val), the training sample set and the validation sample set can be split into N datasets (dataset Tr / Val 1 to dataset Tr / Val n), each dataset including a part of the training sample set and a part of the validation sample set.

[0182] Furthermore, N initial models are constructed, each with different model weights. Temperature prediction models are obtained by training the corresponding initial models on each dataset using the model training method provided in this application embodiment. The model weights of the temperature prediction models obtained from each dataset are M1 to Mn.

[0183] Furthermore, the final model weights M1 to Mn of each temperature prediction model can be averaged to obtain the target model weight Mf, i.e. , where i is the model index.

[0184] Based on this, the new temperature prediction model constructed according to the model weights Mf can be tested and evaluated. Then, the model weights Mf can be adjusted and optimized according to the evaluation results to obtain the target model weights K. At this point, the temperature prediction model constructed according to the target model weights K can be determined as the final temperature prediction model for application.

[0185] When testing and evaluating new temperature models, a separate test set can be used to verify their performance on unknown data. Evaluation metrics typically include accuracy, precision, recall, and F1 score.

[0186] The device management method provided in this application can be applied to an edge-to-cloud data processing and analysis architecture. This architecture can provide complete data processing and analysis support for training temperature prediction models and ensure that the temperature prediction models can play their maximum value in practical applications. After collecting enough data and preprocessing and aggregating it, the data can be used to train the temperature prediction model. The trained temperature prediction model can be deployed to an edge / fog computing environment to quickly respond to data changes in real-time scenarios. Through network services, the trained temperature prediction model can also be accessed and used by other systems or terminals.

[0187] For example, the device management method provided in the embodiments of this application can be applied to Figure 14 In the data processing and analysis architecture shown, Figure 14 This is a schematic diagram of a data processing and analysis architecture implemented in an embodiment of this application.

[0188] High-performance computing (HPC) resources are a comprehensive collection of resources designed specifically for performing large-scale, complex computing tasks. These resources typically include hardware resources, software resources, and cloud HPC resources.

[0189] Data analytics: After data processing and aggregation, data analysis is performed to extract valuable information and insights. This may include various techniques such as statistical analysis, machine learning, and deep learning.

[0190] Storage: Throughout the process, data needs to be securely stored so that it can be accessed and further analyzed at any time.

[0191] Web-Services: Through web services, analysis results or data interfaces can be provided to other systems or terminals, enabling data sharing and integration.

[0192] Edge / Fog Computing: Edge computing and fog computing enable data processing and analysis closer to the data source or endpoint, reducing data transmission latency and improving response speed. Edge computing refers to pushing computing tasks, data storage, and application services from the central node to the edge nodes of the network, i.e., near the data source. These edge nodes can be any device with computing capabilities, such as smartphones, sensors, routers, etc. Edge computing improves the real-time responsiveness and efficiency of the system by reducing data transmission distance and latency. Fog computing is an extension or broader concept of edge computing, further pushing computing and data storage capabilities closer to the data source. Fog computing creates a more distributed and flexible computing architecture by inserting one or more intermediate layers (called fog nodes or fog layers) between the edge and the cloud. These fog nodes can process data from edge devices and perform preliminary analysis and filtering before the data reaches the cloud.

[0193] Aggregation: Based on the collected data, the aggregation step merges similar or related data to reduce the amount of data and improve the efficiency of subsequent processing.

[0194] Collection: The collection process is responsible for the initial processing of the data acquired from the sensors, such as data cleaning and format conversion, to ensure that the data is suitable for subsequent analysis and processing.

[0195] Environment: This is the starting point of the entire process, including various environments and conditions in the physical world, which are monitored by sensors.

[0196] Sensors: Sensors deployed in the environment are responsible for collecting raw data, which may include various types of information such as temperature, humidity, pressure, images, and sound.

[0197] Actuators: Based on the results of data analysis, actuators can perform corresponding physical operations, such as controlling temperature and adjusting brightness.

[0198] End-users: Ultimately, the results of data analysis will be presented to end-users in some form (such as reports, dashboards, applications, etc.) to help them make decisions or take action.

[0199] East-West Bound: This typically refers to the transfer of data within or across data centers, ensuring that data can flow smoothly between different components.

[0200] North-South Bound: refers to the data transfer between data centers and edge computing.

[0201] In this embodiment of the application, after the temperature prediction model is trained, it can be deployed to achieve [the desired result]. Figure 2 The equipment management method shown in steps S21 and S22. See also Figure 15 , Figure 15 This is a schematic diagram of the model deployment process provided in an embodiment of this application. For example... Figure 15 As shown, the model deployment process mainly includes inference device selection, model conversion, edge deployment, deep optimization, and algorithm SDK integration.

[0202] Inference device selection is a crucial step in the deployment of deep learning models, as it determines the model's performance and efficiency in a real-world operating environment. Several key factors need to be considered when selecting a device: Performance Requirements: Select the appropriate inference device based on the application scenario's requirements for computing performance, power consumption, latency, etc. For example, for scenarios requiring high computing performance, a GPU or Neural-network Processing Unit (NPU) can be selected; for power-sensitive scenarios, an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA) may need to be considered.

[0203] Cost-effectiveness: While meeting performance requirements, the cost of components should be considered, including purchase cost, operating cost, and maintenance cost.

[0204] Ecosystem support: Select inference devices with comprehensive ecosystem support, including development tools, model optimization tools, and community support, to facilitate better model deployment and optimization.

[0205] Compatibility: Ensure that the selected inference device is compatible with existing systems or platforms to avoid additional adaptation work.

[0206] Model conversion is the process of transforming a trained temperature prediction model from one format or framework to another for operation on a target inference device. Model conversion typically involves the following steps: Exporting the model: Use tools provided by training frameworks (such as TensorFlow and PyTorch) to export the model as an intermediate representation (IR) format, such as Open Neural Network Exchange (ONNX).

[0207] Model optimization: Perform graph optimization, operator fusion, quantization and other operations on the exported model to improve the model's running efficiency on the target inference device.

[0208] Format conversion: Convert the optimized model into a format supported by the target inference device, such as TensorRT, AscendAI Processor, etc.

[0209] The purpose of model conversion is to ensure that the model can run efficiently on the target inference device while maintaining the model's accuracy and performance.

[0210] Edge deployment refers to the process of deploying deep learning models onto terminal devices (such as mobile phones, cameras, sensors, etc.). Edge deployment offers advantages such as low latency, data privacy protection, and saving cloud resources. Edge deployment typically includes the following steps: Choose an inference engine: Select the appropriate inference engine based on the target device and performance requirements, such as TensorFlow Lite (for Android and iOS) or NCNN (NVIDIA CUDAConvolutional Neural Network) for mobile and embedded devices.

[0211] Deploy the model: Deploy the converted model file to the target device and configure the relevant parameters and settings.

[0212] Testing and verification: Conduct testing and verification on the target device to ensure that the model can run correctly and meet performance requirements.

[0213] Deep optimization refers to further optimizing the temperature prediction model deployed on the edge to improve its operational efficiency and performance. Deep optimization typically includes the following aspects: Quantization: Convert the weights and activation values ​​of the temperature prediction model from floating-point numbers to integers or low-precision floating-point numbers to reduce computational load and storage requirements.

[0214] Pruning: Remove redundant neurons or connections in the temperature prediction model to reduce computation and model size.

[0215] Dynamic adjustment: The computational load and accuracy of the temperature prediction model are dynamically adjusted according to the needs of the actual application scenario to achieve the best balance between performance and power consumption.

[0216] Memory optimization: Optimize memory usage for the temperature prediction model to reduce memory consumption and memory access latency.

[0217] Algorithm software development kit (SDK) integration refers to integrating deep learning algorithms with the SDK to facilitate their use in applications. Algorithm SDK integration typically involves the following steps: Choose an SDK: Select the appropriate SDK based on the target platform and development language.

[0218] SDK Integration: Integrate the SDK into the application's development environment and configure the relevant parameters and settings.

[0219] Calling the algorithm: In the application, the deep learning algorithm is called through the API provided by the SDK to achieve specific functions.

[0220] Testing and verification: Perform testing and verification within the application to ensure that the algorithm runs correctly and meets performance requirements.

[0221] Algorithm SDK integration makes it easier to apply deep learning algorithms to real-world scenarios, improving development efficiency and the intelligence level of applications.

[0222] The training sample set, validation sample set, and test set in this application embodiment can be stored in the target storage space associated with the target device. The target storage space includes, but is not limited to, cloud storage, database, blockchain, etc. The specific storage space can be determined based on the actual application scenario requirements and is not limited here.

[0223] When the palm, face (or other biometric) recognition technologies involved in the embodiments of this application are applied to specific products or technologies, the relevant data collection, use and processing processes should comply with the requirements of national laws and regulations. Before collecting biometric information, the information processing rules should be communicated and the individual consent of the target object should be obtained. Biometric information should be processed in strict accordance with the requirements of laws and regulations and personal information processing rules, and technical measures should be taken to ensure the security of relevant data.

[0224] See Figure 16 , Figure 16 This is a schematic diagram of the device management apparatus provided in an embodiment of this application. The device management apparatus provided in an embodiment of this application includes: The image acquisition module 161 is used to acquire an infrared thermal image of the target device and perform image segmentation processing on the infrared thermal image to obtain multiple sub-images; Temperature prediction module 162 is used to input each sub-image of the infrared thermal image into the temperature prediction model to obtain the predicted temperature value of the infrared thermal image. The equipment management module 163 is used to manage the working status of the target equipment based on the predicted temperature value of the target equipment. The temperature prediction model described above is constructed based on a gated recurrent unit (GRU) network, and the predicted temperature value is determined based on the following method: Based on each sub-image of the infrared thermal image, multiple feature units are determined, and the feature units are fused to obtain the temperature features of the infrared thermal image. Each feature unit is used to characterize the temperature features of one sub-image. Based on the temperature characteristics of the infrared thermal image, the predicted temperature value of the target device is determined.

[0225] In some feasible implementations, the above-mentioned fusion of the aforementioned feature units to obtain the temperature features of the infrared thermal image includes: Determine the temperature information of at least one hardware component of the aforementioned target device; Attention characteristics are determined based on the temperature information of each hardware component; Based on the above attention features and each of the above feature units, a corresponding first fusion feature is determined; The temperature features of the infrared thermal image are determined based on the first fusion feature corresponding to each of the aforementioned feature units.

[0226] In some feasible implementations, determining the temperature features of the infrared thermal image based on the first fusion feature corresponding to each of the aforementioned feature units includes: Based on the position information of each sub-image of the infrared thermal image in the infrared thermal image, each of the first fusion features is stitched together to obtain the temperature features of the infrared thermal image.

[0227] In some feasible implementations, determining the predicted temperature value of the target device based on the temperature characteristics of the infrared thermal image includes: The temperature features of the infrared thermal images are processed by a convolutional network to obtain convolutional features; The temperature features described above are processed by a GRU network to obtain intermediate features, and attention fusion features are determined based on these intermediate features. The convolutional features and the attention fusion features described above are fused to obtain the second fusion feature; Based on the second fusion feature mentioned above, the predicted temperature value of the target device is determined.

[0228] In some feasible implementations, the above temperature prediction model is trained by the training device in the following manner: A training sample set and a validation sample set are determined. The training sample set includes multiple first image sets, each of which includes multiple sub-images of a first infrared thermal image. The validation sample set includes multiple second image sets, each of which includes multiple sub-images of a second infrared thermal image. The initial model is trained iteratively a predetermined number of times based on the aforementioned training sample set; In each iteration, each of the aforementioned first image sets is input into the initial model obtained in the previous iteration to obtain the predicted temperature value of the target device corresponding to each of the aforementioned first infrared thermal images; a first training loss is determined based on the actual temperature value and predicted temperature value of the target device corresponding to each of the aforementioned first infrared thermal images; and the initial model obtained in the previous iteration is adjusted based on the aforementioned first training loss to obtain the initial model obtained in the current iteration. After each iteration, each set of the second images is input into the initial model obtained in this iteration to obtain the predicted temperature value of the target device corresponding to each of the second infrared thermal images. Based on the actual temperature value and the predicted temperature value of the target device corresponding to each of the second infrared thermal images, the first verification loss is determined. Based on the first loss set and the second loss set, the above temperature prediction model is determined from the initial model obtained in each iteration process; The first loss set includes the first training loss obtained in each iteration, and the second loss set includes the first verification loss obtained after each iteration.

[0229] In some feasible implementations, when the training device determines the temperature prediction model from the initial model obtained in each iteration based on the first loss set and the second loss set, it is used to: The verification error curve is obtained by smoothing the first verification loss in the second loss set mentioned above. In response to the presence of outliers in the verification error curve that meet the preset conditions, the initial model obtained from the iterative process corresponding to the outlier is determined as the temperature prediction model. In response to the absence of outliers in the verification error curve that meet the preset conditions, the first training loss in the first loss set is smoothed to obtain the training error curve. In response to the intersection of the training error curve and the verification error curve, the initial model obtained from the iterative process corresponding to the intersection point is determined as the temperature prediction model; in response to the non-intersection of the training error curve and the verification error curve, the temperature prediction model is determined from the initial model obtained from each iteration based on the training error curve.

[0230] In some feasible implementations, when the training device determines the temperature prediction model from the initial model obtained in each iteration based on the training error curve, it is used to: In response to the fact that the training error curve has not reached the convergence condition, the initial model obtained in the Mth iteration is determined as the temperature prediction model. The Mth iteration is the iteration process corresponding to the minimum value in the training error curve. In response to the convergence of the training error curve, the initial model obtained by the iteration process with the fewest iterations between the Mth iteration process and the Nth iteration process that makes the training error curve converge is determined as the temperature prediction model.

[0231] In some feasible implementations, when the image acquisition module 161 performs image block processing on the infrared thermal image to obtain multiple sub-images, it is used to: The neighborhood window for each pixel in the infrared thermal image is determined based on a preset neighborhood window. For each of the above pixels, determine the median value of the original pixel values ​​of all other pixels in the neighborhood window of the pixel, and replace the original pixel value of the pixel according to the median value. The denoised image is determined based on each pixel after its value has been replaced, and the denoised image is then processed into multiple sub-images by image block processing.

[0232] In some feasible implementations, the target device is used to identify the target object, and when the device management module 163 manages the operating status of the target device based on the predicted temperature value of the target device, it is used to: In response to the fact that the predicted temperature value of the target device is higher than the preset temperature value, and there is no target object approaching the target device within the recognition distance range of the target device, the target device is controlled to operate in the first working mode. In response to the predicted temperature value of the target device being higher than the preset temperature value and the presence of a target object approaching the target device within the identification distance range, or in response to the predicted temperature value of the target device being lower than the preset temperature value, the target device is controlled to operate in a second working mode. The power consumption of the target device in the first working mode is less than that in the second working mode.

[0233] In some feasible implementations, when the device management module 163 determines whether there is a target object approaching the target device within the identification distance range, it is used to: In response to the appearance of a target object within the aforementioned identification distance range, a first data sequence is determined, wherein the first data sequence includes distance information between the target device and the target object obtained at preset time intervals; The first data sequence is input into the pre-trained behavior prediction model to obtain the behavior prediction result of the target object. Based on the behavior prediction result, it is determined whether the target object is approaching the target device.

[0234] In some feasible implementations, the device management module 163 is further configured to: In response to the above behavior prediction result meeting the preset conditions and the above target device being in a sleep state, the above target device is controlled to enter a wake-up state; in response to the above behavior prediction result not meeting the above preset conditions and the above target device being in a wake-up state, the above target device is controlled to enter a sleep state. Determine the operating frequency that matches the above behavior prediction results, and control the target equipment to operate at the matching operating frequency; The aforementioned behavioral prediction results include at least one of the following: The behavioral state of the aforementioned target object includes approaching or moving away from the aforementioned target device; The approach speed of the aforementioned target object; The speed at which the aforementioned target object moves away; The duration of stay for the aforementioned target individuals; The aforementioned behavioral prediction results meet the aforementioned preset conditions, including at least one of the following: The aforementioned target object is close to the aforementioned target device; The approach speed of the target object is greater than the preset approach speed; The dwell time of the aforementioned target objects is longer than the preset dwell time; The above behavioral prediction results do not meet the above preset conditions, including at least one of the following: The aforementioned target object is located far away from the aforementioned target equipment; The aforementioned target object's moving away speed is greater than the preset moving away speed; The dwell time of the aforementioned target object is less than or equal to the aforementioned preset dwell time.

[0235] In some feasible implementations, when the behavior prediction result includes the behavior state, the operating frequency of the target device when the target object is close to the target device is greater than the operating frequency when the target object is far away from the target device. When the above behavior prediction results include the above approach speed, the operating frequency of the above target device is positively correlated with the above approach speed; When the above behavior prediction results include the above dwell time, the operating frequency of the above target device is negatively correlated with the above dwell time. When the above behavior prediction results include the above-mentioned distance speed, the operating frequency of the above-mentioned target device is negatively correlated with the above-mentioned distance speed.

[0236] In practice, the aforementioned device management device can perform the above-described actions through its built-in functional modules. Figure 2 The implementation methods provided for each step are detailed in the above-mentioned implementation methods, and will not be repeated here.

[0237] See Figure 17 , Figure 17 This is a schematic diagram of the structure of the electronic device provided in an embodiment of this application. For example... Figure 17 As shown, the electronic device 1700 in this embodiment may include: a processor 1701, a network interface 1704, and a memory 1705. Furthermore, the electronic device 1700 may also include: an object interface 1703, and at least one communication bus 1702. The communication bus 1702 is used to implement communication between these components. The object interface 1703 may include a display screen and a keyboard; optionally, the object interface 1703 may also include a standard wired interface or a wireless interface. The network interface 1704 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1705 may be a high-speed RAM or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory 1705 may also be at least one storage device located remotely from the aforementioned processor 1701. Figure 17 As shown, the memory 1705, which is a computer-readable storage medium, may include an operating system, a network communication module, an object interface module, and a device control application.

[0238] exist Figure 17 In the illustrated electronic device 1700, the network interface 1704 provides network communication functionality; the object interface 1703 is primarily used to provide an input interface for objects; and the processor 1701 can be used to call the device control application stored in the memory 1705 to achieve: Acquire an infrared thermal image of the target device, and perform image segmentation processing on the infrared thermal image to obtain multiple sub-images; By inputting each sub-image of the infrared thermal image into the temperature prediction model, the predicted temperature value of the infrared thermal image is obtained. The operating status of the target equipment is managed based on the predicted temperature value of the target equipment. The temperature prediction model described above is constructed based on a gated recurrent unit (GRU) network, and the predicted temperature value is determined based on the following method: Based on each sub-image of the infrared thermal image, multiple feature units are determined, and the feature units are fused to obtain the temperature features of the infrared thermal image. Each feature unit is used to characterize the temperature features of one sub-image. Based on the temperature characteristics of the infrared thermal image, the predicted temperature value of the target device is determined.

[0239] In some feasible implementations, when the processor 1701 fuses the aforementioned feature units to obtain the temperature features of the aforementioned infrared thermal image, it is used to: Determine the temperature information of at least one hardware component of the aforementioned target device; Attention characteristics are determined based on the temperature information of each hardware component; Based on the above attention features and each of the above feature units, a corresponding first fusion feature is determined; The temperature features of the infrared thermal image are determined based on the first fusion feature corresponding to each of the aforementioned feature units.

[0240] In some feasible implementations, when the processor 1701 determines the temperature features of the infrared thermal image based on the first fusion feature corresponding to each of the aforementioned feature units, it is used to: Based on the position information of each sub-image of the infrared thermal image in the infrared thermal image, each of the first fusion features is stitched together to obtain the temperature features of the infrared thermal image.

[0241] In some feasible implementations, when the processor 1701 determines the predicted temperature value of the target device based on the temperature characteristics of the infrared thermal image, it is used to: The temperature features of the infrared thermal images are processed by a convolutional network to obtain convolutional features; The temperature features described above are processed by a GRU network to obtain intermediate features, and attention fusion features are determined based on these intermediate features. The convolutional features and the attention fusion features described above are fused to obtain the second fusion feature; Based on the second fusion feature mentioned above, the predicted temperature value of the target device is determined.

[0242] In some feasible implementations, the temperature prediction model described above is trained by the processor 1701 in the following manner: A training sample set and a validation sample set are determined. The training sample set includes multiple first image sets, each of which includes multiple sub-images of a first infrared thermal image. The validation sample set includes multiple second image sets, each of which includes multiple sub-images of a second infrared thermal image. The initial model is trained iteratively a predetermined number of times based on the aforementioned training sample set; In each iteration, each of the aforementioned first image sets is input into the initial model obtained in the previous iteration to obtain the predicted temperature value of the target device corresponding to each of the aforementioned first infrared thermal images; a first training loss is determined based on the actual temperature value and predicted temperature value of the target device corresponding to each of the aforementioned first infrared thermal images; and the initial model obtained in the previous iteration is adjusted based on the aforementioned first training loss to obtain the initial model obtained in the current iteration. After each iteration, each set of the second images is input into the initial model obtained in this iteration to obtain the predicted temperature value of the target device corresponding to each of the second infrared thermal images. Based on the actual temperature value and the predicted temperature value of the target device corresponding to each of the second infrared thermal images, the first verification loss is determined. Based on the first loss set and the second loss set, the above temperature prediction model is determined from the initial model obtained in each iteration process; The first loss set includes the first training loss obtained in each iteration, and the second loss set includes the first verification loss obtained after each iteration.

[0243] In some feasible implementations, when the processor 1701 determines the temperature prediction model from the initial model obtained in each iteration based on the first loss set and the second loss set, it is used to: The verification error curve is obtained by smoothing the first verification loss in the second loss set mentioned above. In response to the presence of outliers in the verification error curve that meet the preset conditions, the initial model obtained from the iterative process corresponding to the outlier is determined as the temperature prediction model. In response to the absence of outliers in the verification error curve that meet the preset conditions, the first training loss in the first loss set is smoothed to obtain the training error curve. In response to the intersection of the training error curve and the verification error curve, the initial model obtained from the iterative process corresponding to the intersection point is determined as the temperature prediction model; in response to the non-intersection of the training error curve and the verification error curve, the temperature prediction model is determined from the initial model obtained from each iteration based on the training error curve.

[0244] In some feasible implementations, when the processor 1701 determines the temperature prediction model from the initial model obtained in each iteration based on the training error curve, it is used to: In response to the fact that the training error curve has not reached the convergence condition, the initial model obtained in the Mth iteration is determined as the temperature prediction model. The Mth iteration is the iteration process corresponding to the minimum value in the training error curve. In response to the convergence of the training error curve, the initial model obtained by the iteration process with the fewest iterations between the Mth iteration process and the Nth iteration process that makes the training error curve converge is determined as the temperature prediction model.

[0245] In some feasible implementations, when the processor 1701 performs image block processing on the infrared thermal image to obtain multiple sub-images, it is used to: The neighborhood window for each pixel in the infrared thermal image is determined based on a preset neighborhood window. For each of the above pixels, determine the median value of the original pixel values ​​of all other pixels in the neighborhood window of the pixel, and replace the original pixel value of the pixel according to the median value. The denoised image is determined based on each pixel after its value has been replaced, and the denoised image is then processed into multiple sub-images by image block processing.

[0246] In some feasible implementations, the target device is used to identify a target object, and when the processor 1701 manages the operating status of the target device based on the predicted temperature value of the target device, it is used to: In response to the fact that the predicted temperature value of the target device is higher than the preset temperature value, and there is no target object approaching the target device within the recognition distance range of the target device, the target device is controlled to operate in the first working mode. In response to the predicted temperature value of the target device being higher than the preset temperature value and the presence of a target object approaching the target device within the identification distance range, or in response to the predicted temperature value of the target device being lower than the preset temperature value, the target device is controlled to operate in a second working mode. The power consumption of the target device in the first working mode is less than that in the second working mode.

[0247] In some feasible implementations, when the processor 1701 determines whether there is a target object approaching the target device within the recognition distance range, it is used to: In response to the appearance of a target object within the aforementioned identification distance range, a first data sequence is determined, wherein the first data sequence includes distance information between the target device and the target object obtained at preset time intervals; The first data sequence is input into the pre-trained behavior prediction model to obtain the behavior prediction result of the target object. Based on the behavior prediction result, it is determined whether the target object is approaching the target device.

[0248] In some feasible implementations, the processor 1701 is further used for: In response to the above behavior prediction result meeting the preset conditions and the above target device being in a sleep state, the above target device is controlled to enter a wake-up state; in response to the above behavior prediction result not meeting the above preset conditions and the above target device being in a wake-up state, the above target device is controlled to enter a sleep state. Determine the operating frequency that matches the above behavior prediction results, and control the target equipment to operate at the matching operating frequency; The aforementioned behavioral prediction results include at least one of the following: The behavioral state of the aforementioned target object includes approaching or moving away from the aforementioned target device; The approach speed of the aforementioned target object; The speed at which the aforementioned target object moves away; The duration of stay for the aforementioned target individuals; The aforementioned behavioral prediction results meet the aforementioned preset conditions, including at least one of the following: The aforementioned target object is close to the aforementioned target device; The approach speed of the target object is greater than the preset approach speed; The dwell time of the aforementioned target objects is longer than the preset dwell time; The above behavioral prediction results do not meet the above preset conditions, including at least one of the following: The aforementioned target object is located far away from the aforementioned target equipment; The aforementioned target object's moving away speed is greater than the preset moving away speed; The dwell time of the aforementioned target object is less than or equal to the aforementioned preset dwell time.

[0249] In some feasible implementations, when the behavior prediction result includes the behavior state, the operating frequency of the target device when the target object is close to the target device is greater than the operating frequency when the target object is far away from the target device. When the above behavior prediction results include the above approach speed, the operating frequency of the above target device is positively correlated with the above approach speed; When the above behavior prediction results include the above dwell time, the operating frequency of the above target device is negatively correlated with the above dwell time. When the above behavior prediction results include the above-mentioned distance speed, the operating frequency of the above-mentioned target device is negatively correlated with the above-mentioned distance speed.

[0250] It should be understood that in some feasible implementations, the processor 1701 described above may be a central processing unit (CPU), which may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. The memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.

[0251] In specific implementation, the aforementioned electronic device 1700 can perform the above-described actions through its built-in functional modules. Figure 2 The implementation methods provided for each step are detailed in the above-mentioned implementation methods, and will not be repeated here.

[0252] This application also provides a computer-readable storage medium storing a computer program that is executed by a processor to implement... Figure 2 The methods provided in each step are detailed in the implementation methods provided in the above steps, and will not be repeated here.

[0253] The aforementioned computer-readable storage medium can be an internal storage unit of the device management apparatus or electronic device provided in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device. The aforementioned computer-readable storage medium can also include magnetic disks, optical disks, read-only memory (ROM), or random access memory (RAM), etc. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of the electronic device. The computer-readable storage medium is used to store the computer program and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0254] This application provides a computer program product, which includes a computer program that is executed by a processor. Figure 2 The methods provided for each step in the process.

[0255] The terms "first," "second," etc., used in the claims, description, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or electronic device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or electronic devices. References to "embodiment" herein mean that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The presentation of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. The term "and / or" as used in this application's description and appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0256] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Those skilled in the art can implement the described functions using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.

[0257] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A method for managing equipment, characterized in that, The method includes: Acquire an infrared thermal image of the target device, and perform image segmentation processing on the infrared thermal image to obtain multiple sub-images; Each sub-image of the infrared thermal image is input into the temperature prediction model to obtain the predicted temperature value of the infrared thermal image. The operating status of the target device is managed based on the predicted temperature value of the target device; The temperature prediction model is constructed based on a gated recurrent unit (GRU) network, and the predicted temperature value is determined based on the following method: Multiple feature units are determined based on each sub-image of the infrared thermal image, and the feature units are fused to obtain the temperature features of the infrared thermal image. Each feature unit is used to characterize the temperature features of one sub-image. Based on the temperature characteristics of the infrared thermal image, the predicted temperature value of the target device is determined.

2. The method according to claim 1, characterized in that, The process of fusing the aforementioned feature units to obtain the temperature features of the infrared thermal image includes: Determine the temperature information of at least one hardware component of the target device; Attention characteristics are determined based on the temperature information of each hardware component; A corresponding first fusion feature is determined based on the attention features and each of the feature units; The temperature features of the infrared thermal image are determined based on the first fusion feature corresponding to each feature unit.

3. The method according to claim 2, characterized in that, Determining the temperature features of the infrared thermal image based on the first fusion feature corresponding to each feature unit includes: Based on the position information of each sub-image in the infrared thermal image, each of the first fusion features is stitched together to obtain the temperature features of the infrared thermal image.

4. The method according to claim 1, characterized in that, Determining the predicted temperature value of the target device based on the temperature characteristics of the infrared thermal image includes: The temperature features of the infrared thermal image are processed by a convolutional network to obtain convolutional features; The temperature features are processed by a GRU network to obtain intermediate features, and attention fusion features are determined based on the intermediate features. The convolutional features and the attention fusion features are fused to obtain the second fusion feature; Based on the second fusion feature, the predicted temperature value of the target device is determined.

5. The method according to claim 1, characterized in that, The temperature prediction model was trained using the following method: A training sample set and a validation sample set are determined. The training sample set includes multiple first image sets, each of which includes multiple sub-images of a first infrared thermal image. The validation sample set includes multiple second image sets, each of which includes multiple sub-images of a second infrared thermal image. The initial model is trained iteratively a predetermined number of times based on the training sample set; In each iteration, each set of the first images is input into the initial model obtained in the previous iteration to obtain the predicted temperature value of the target device corresponding to each of the first infrared thermal images; The first training loss is determined based on the actual temperature value and the predicted temperature value of the target device corresponding to each first infrared thermal image. The initial model obtained in the previous iteration is adjusted based on the first training loss to obtain the initial model obtained in the current iteration. After each iteration, each set of the second images is input into the initial model obtained in this iteration to obtain the predicted temperature value of the target device corresponding to each second infrared thermal image. Based on the actual temperature value and the predicted temperature value of the target device corresponding to each second infrared thermal image, the first verification loss is determined. The temperature prediction model is determined from the initial model obtained in each iteration based on the first loss set and the second loss set. The first loss set includes the first training loss obtained in each iteration, and the second loss set includes the first verification loss obtained after each iteration.

6. The method according to claim 5, characterized in that, The step of determining the temperature prediction model from the initial model obtained in each iteration based on the first loss set and the second loss set includes: The first verification loss in the second loss set is smoothed to obtain the verification error curve; In response to the presence of outliers in the verification error curve that meet preset conditions, the initial model obtained from the iterative process corresponding to the outlier is determined as the temperature prediction model; in response to the absence of outliers in the verification error curve that meet the preset conditions, the first training loss in the first loss set is smoothed to obtain the training error curve. In response to the intersection of the training error curve and the verification error curve, the initial model obtained from the iteration process corresponding to the intersection point is determined as the temperature prediction model; in response to the non-intersection of the training error curve and the verification error curve, the temperature prediction model is determined from the initial model obtained from each iteration process based on the training error curve.

7. The method according to claim 6, characterized in that, The step of determining the temperature prediction model from the initial model obtained in each iteration based on the training error curve includes: In response to the fact that the training error curve has not reached the convergence condition, the initial model obtained in the Mth iteration process is determined as the temperature prediction model, and the Mth iteration process is the iteration process corresponding to the minimum value in the training error curve; In response to the convergence of the training error curve, the initial model obtained by the iteration process with the fewest iterations between the Mth iteration process and the Nth iteration process that makes the training error curve converge is determined as the temperature prediction model.

8. The method according to claim 1, characterized in that, The process of segmenting the infrared thermal image into multiple sub-images includes: The neighborhood window for each pixel of the infrared thermal image is determined according to a preset neighborhood window. For each pixel, determine the median value of the original pixel values ​​of all other pixels within the neighborhood window of that pixel, and replace the original pixel value of that pixel based on the median value; The denoised image is determined based on each pixel after its value has been replaced, and the denoised image is then processed into multiple sub-images by image block processing.

9. The method according to claim 1, characterized in that, The target device is used to identify target objects, and the management of the operating status of the target device based on its predicted temperature value includes: In response to the predicted temperature value of the target device being higher than the preset temperature value, and the absence of any target object approaching the target device within the recognition distance range of the target device, the target device is controlled to operate in a first working mode; In response to the predicted temperature value of the target device being higher than the preset temperature value and the presence of a target object approaching the target device within the recognition distance range, or in response to the predicted temperature value of the target device being lower than the preset temperature value, the target device is controlled to operate in a second working mode. Wherein, the power consumption of the target device in the first working mode is less than the power consumption of the device in the second working mode.

10. The method according to claim 9, characterized in that, Determining whether there is a target object approaching the target device within the recognition distance range includes: In response to the appearance of a target object within the recognition distance range, a first data sequence is determined, the first data sequence including distance information between the target device and the target object obtained at preset time intervals; The first data sequence is input into a pre-trained behavior prediction model to obtain the behavior prediction result of the target object, and the target object is determined to move closer to the target device based on the behavior prediction result.

11. The method according to claim 10, characterized in that, The method further includes at least one of the following: In response to the behavior prediction result meeting the preset conditions and the target device being in a sleep state, the target device is controlled to enter a wake-up state; in response to the behavior prediction result not meeting the preset conditions and the target device being in a wake-up state, the target device is controlled to enter a sleep state. Determine the operating frequency that matches the behavior prediction result, and control the target device to operate at the matching operating frequency; The behavior prediction result includes at least one of the following: The behavioral state of the target object includes approaching the target device or moving away from the target device; The approach speed of the target object; The speed at which the target object moves away; The dwell time of the target object; Wherein, the behavior prediction result meets the preset conditions, including at least one of the following: The target object is close to the target device; The approach speed of the target object is greater than the preset approach speed; The dwell time of the target object is greater than the preset dwell time; The behavior prediction result does not meet the preset conditions, including at least one of the following: The target object is moved away from the target device; The target object's moving away speed is greater than the preset moving away speed; The dwell time of the target object is less than or equal to the preset dwell time.

12. The method according to claim 11, characterized in that, When the behavior prediction result includes the behavior state, the operating frequency of the target device when the target object is close to the target device is greater than the operating frequency when the target object is far away from the target device; When the behavior prediction result includes the approach speed, the operating frequency of the target device is positively correlated with the approach speed; When the behavior prediction result includes the dwell time, the operating frequency of the target device is negatively correlated with the dwell time; When the behavior prediction result includes the distance speed, the operating frequency of the target device is negatively correlated with the distance speed.

13. An equipment management device, characterized in that, The device includes: The image acquisition module is used to acquire an infrared thermal image of the target device and perform image block processing on the infrared thermal image to obtain multiple sub-images; The temperature prediction module is used to input each sub-image of the infrared thermal image into the temperature prediction model to obtain the predicted temperature value of the infrared thermal image. The equipment management module is used to manage the operating status of the target equipment based on the predicted temperature value of the target equipment; The temperature prediction model is constructed based on a gated recurrent unit (GRU) network, and the predicted temperature value is determined based on the following method: Multiple feature units are determined based on each sub-image of the infrared thermal image, and the feature units are fused to obtain the temperature features of the infrared thermal image. Each feature unit is used to characterize the temperature features of one sub-image. Based on the temperature characteristics of the infrared thermal image, the predicted temperature value of the target device is determined.

14. An electronic device, characterized in that, It includes a processor and a memory, which are interconnected; The memory is used to store computer programs; The processor is configured to execute the method according to any one of claims 1 to 12 when the computer program is invoked.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is executed by a processor to implement the method of any one of claims 1 to 12.

16. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 12.

Images