Radar and image data fused abnormal check-in state identification method

Abstract

The invention relates to the technical field of intelligent safety management, and discloses an abnormal check-in state identification method fusing radar and image data, and the method comprises the following steps: S1, obtaining check-in order information, and configuring a check-in rule which comprises a room number, a preset number of people, a no-smoking area and no-area setting; s2, acquiring personnel data in real time through millimeter wave radar equipment and image acquisition equipment, wherein the personnel data comprises the number, the position and the behavior state of personnel; s3, preprocessing the collected millimeter wave radar data and image data, wherein the preprocessing comprises denoising, standardization and time synchronization processing; and S4, performing multi-source fusion on the preprocessed millimeter wave radar data and image data. Through a multi-source acquisition and intelligent detection technology fusing millimeter wave radar and image data, the number of people in a room and behavior characteristics in a public area are accurately detected, the detection accuracy and sensitivity can be remarkably improved even in a complex environment, and the misinformation and missing report risks are reduced.

Description

Technical Field

[0001] The present invention relates to the field of intelligent security management technology, and in particular to a method for identifying abnormal occupancy status by integrating radar and image data. Background Art

[0002] With the development of society and the continuous advancement of technology, people have higher requirements for the safety and privacy of their living environments. This is especially true in places like hotels, accommodation and hospitality venues, homestays, and online rental properties. Security management has become a crucial issue. Traditional security management methods rely primarily on video surveillance and access control systems. While these methods have improved security to a certain extent, their limitations are also becoming increasingly apparent. Faced with increasingly complex security needs and demands for guest privacy, these traditional technologies cannot fully address all safety hazards, easily creating regulatory blind spots and failing to meet the security management needs of the modern accommodation and hospitality industry.

[0003] Security management systems based on the Internet of Things and artificial intelligence (AI) are becoming increasingly mainstream. These systems collect environmental data in real time through sensor networks and analyze and process this data using intelligent algorithms, enabling intelligent monitoring and control of human activity. In hotel management scenarios, these technologies can effectively improve the safety and operational efficiency of the tourism industry. However, existing single-use video surveillance or access control systems often suffer from numerous flaws, such as inaccurate capture of behavioral details, insufficient privacy protection, and poor monitoring effectiveness in low-light or obstructed conditions. These issues make it difficult to quickly and accurately detect and respond to abnormal behavior (such as overcrowding or entry into restricted areas), increasing management complexity and security risks.

[0004] The present invention proposes an abnormal occupancy status recognition method that integrates radar and image data. Summary of the Invention

[0005] In response to the shortcomings of existing technologies, the present invention provides a method for identifying abnormal occupancy status by integrating radar and image data, which solves the problems in traditional security management that cannot fully cover safety hazards, quickly and accurately detect abnormal occupancy behavior, and protect resident privacy.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: A method for identifying abnormal occupancy status by integrating radar and image data includes the following steps: S1. Obtain check-in order information and configure check-in rules, including room number, preset number of people, no-smoking area, and restricted area settings; S2. Collect personnel data in real time through millimeter-wave radar equipment and image acquisition equipment, including the number of personnel, location, and behavior status; S3. Preprocessing the collected millimeter-wave radar data and image data, wherein the preprocessing includes denoising, standardization, and time synchronization processing; S4. Performing multi-source fusion on the pre-processed millimeter-wave radar data and image data, wherein the data fusion includes weighted fusion and Kalman filtering processing; S5. Detect abnormal conditions based on the data fusion results and preset rules. The abnormal conditions include overcrowding, violations in no-smoking areas, and entry into restricted areas. S6. Generate an abnormal record and send an early warning notification to the management personnel through the management system.

[0007] Preferably, the step S2 specifically includes the following steps: S2.1. Millimeter-wave radar data collection: Deploy millimeter-wave radar equipment in public areas to collect data including the number of people, their location coordinates, and their movement trajectories. S2.2. Image Data Acquisition: Deploy image acquisition equipment to capture real-time image data of public areas. Using computer vision algorithms, extract data including the outlines and number of people and their behavioral characteristics. S2.3. Data time synchronization: Add timestamps to radar data and image data to ensure consistency in the time dimension.

[0008] Preferably, in step S2.1, the millimeter wave radar device transmits electromagnetic waves and receives reflected signals, and calculates the target person's movement trajectory, including distance and speed, based on time delay and Doppler frequency shift, wherein: The distance calculation formula is: Where d is the target distance, c is the speed of light, and t is the round-trip time difference of the signal; The speed calculation formula is: Where v is the target speed, f d is the frequency shift, and λ is the millimeter wave wavelength.

[0009] Preferably, the multi-source fusion of the millimeter-wave radar data and the image data in step S4 is achieved by a weighted fusion algorithm, and the fusion weight in the weighted fusion algorithm is dynamically adjusted according to the data confidence, and the calculation formula is: W 融合 =αW 雷达 +(1-α)W 图像 Among them, α is the weight of millimeter wave radar data, and 1-α is the weight of image data.

[0010] Preferably, the abnormal state detection in step S5 includes: Number of people anomaly detection: the real-time number of people N detected融合 With the preset number N 预设 Compare, when ΔN=N 融合 -N 预设 When >0, it is determined that the number of people is abnormal; Behavioral anomaly detection: Detect violations such as smoking or entering restricted areas using image data.

[0011] Preferably, the image data processing in the preprocessing step in step S3 includes: Image enhancement, improving image quality through histogram equalization; Edge detection, using the Canny algorithm to extract the target contour; Background removal reduces irrelevant interference and retains the effective detection area.

[0012] Preferably, the abnormal record in step S6 includes the abnormal type, room number, detection time, number of people in the order and the actual number of people detected.

[0013] Preferably, the millimeter wave radar data preprocessing in step S3 includes: Gaussian filtering denoising to remove random noise; Mean smoothing to reduce errors; Timestamp synchronization to synchronize data with image data.

[0014] The present invention provides a method for identifying abnormal occupancy status by integrating radar and image data. It has the following beneficial effects: 1. This invention effectively overcomes the shortcomings of a single sensor by integrating multi-source data acquisition and fusion technology with millimeter-wave radar data and image data. The millimeter-wave radar can accurately detect the number, location, and movement trajectory of people, while the image acquisition device can identify people's behavioral characteristics and illegal actions. Through a weighted fusion algorithm and Kalman filtering processing, the system dynamically adjusts the weights of the two types of data to ensure accurate data support even in complex environments. In particular, in abnormal situations such as when the number of people in the detection room exceeds the preset number, the fused detection results can significantly improve the accuracy and sensitivity, reduce the risk of false alarms and missed alarms, and ensure the reliability of safety detection.

[0015] 2. The present invention combines data fusion with intelligent detection algorithms to quickly identify abnormal occupancy phenomena such as overcrowding, smoking, or entering restricted areas by collecting and analyzing the number of people in the room and behavioral characteristics in public areas in real time. After detecting an abnormal state, the system sends an early warning notification to the management personnel through multiple channels such as the background management system, SMS, and APP push, enabling the management personnel to intervene and deal with the abnormality as soon as it occurs, reducing safety hazards. In addition, the storage and analysis function of abnormal records provides data support for the optimization of long-term safety management strategies for reception and accommodation venues, thereby further improving the efficiency of safety management and the ability to cope with complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Flowchart of the method of the present invention. DETAILED DESCRIPTION

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the present specification. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0018] Please see the attached Figure 1 , an embodiment of the present invention provides a method for identifying abnormal occupancy status by fusing radar and image data, comprising the following steps: S1. Obtain check-in order information and configure check-in rules, including room number, preset number of people, non-smoking areas, and restricted areas. S2. Collect personnel data in real time through millimeter-wave radar equipment and image acquisition equipment, including the number of personnel, location, and behavior status; S3. Preprocessing the collected millimeter-wave radar data and image data, including denoising, standardization, and time synchronization processing; S4, performing multi-source fusion on the pre-processed millimeter-wave radar data and image data, wherein the data fusion includes weighted fusion and Kalman filtering processing; S5. Detect abnormal conditions based on the data fusion results and preset rules. Abnormal conditions include overcrowding, violations of no-smoking areas, and entry into restricted areas. S6. Generate an abnormal record and send an early warning notification to the management personnel through the management system.

[0019] Specifically, this invention proposes a method for identifying abnormal occupancy states that integrates millimeter-wave radar and image data. Through multi-source data fusion and intelligent detection, this method achieves accurate identification of abnormal conditions and provides early warning feedback. The specific implementation steps are as follows: First, during the check-in phase, the system retrieves check-in order information from the accommodation and reception venue management system, including room number, pre-set number of occupants, check-in time, and other information. It then configures relevant rules based on the accommodation and reception venue's management requirements, such as room type and occupancy restrictions, and the establishment of no-smoking and unauthorized restricted areas. These rules, along with the order information, are stored in a backend database, providing a basis for subsequent anomaly detection.

[0020] After the system is deployed, millimeter-wave radar and image acquisition equipment collect real-time data on people in public areas. The millimeter-wave radar detects the number, location, and movement of people, calculating their distance and speed based on the radar signal's reflection characteristics, using time delay and Doppler shift. Simultaneously, the image acquisition equipment captures image data of public areas, using computer vision algorithms to identify people's outlines and behavioral characteristics, and determine the current number of people and their status. To ensure the synchronization of the two types of data, the system synchronizes the radar and image data using timestamp technology.

[0021] After data collection is complete, the system preprocesses the millimeter-wave radar and image data. The millimeter-wave radar data is denoised using Gaussian filtering and mean smoothing to eliminate noise interference, and the position data is coordinate normalized. Image data is enhanced using image enhancement techniques, edge detection is used to extract person outlines, and background removal is used to reduce interference from irrelevant information. The preprocessed data is timestamped to ensure data consistency and accuracy.

[0022] To improve the accuracy of anomaly detection, the system performs multi-source fusion processing on preprocessed millimeter-wave radar data and image data. First, the system dynamically assigns weights based on the confidence level of the data sources using a weighted fusion algorithm to fuse the radar and image data. Second, it uses a Kalman filter algorithm to estimate and update occupant status, removing data noise and improving detection accuracy. Ultimately, it outputs key information such as the real-time number of people in the room, their location coordinates, and their behavior.

[0023] Based on the data fusion results, the system performs anomaly detection on the status of public areas according to preset rules. The system first detects whether the current actual number of occupants is consistent with the preset number of occupants. If the actual number exceeds the preset number, it is determined to be an overcrowded state. At the same time, the system uses image data to analyze the behavioral characteristics of the guests to detect whether there is smoking behavior or illegal entry into restricted areas. For smoking behavior, the system makes judgments through the recognition of smoke characteristics and hand movements; for restricted area detection, the system compares the position coordinates of the person with the range of the restricted area to determine whether there is illegal entry. In addition, in order to improve the intelligence level of detection, the system introduces deep learning models (such as YOLO or CNN) for auxiliary classification and verification to improve the robustness and accuracy of anomaly detection.

[0024] When the system detects an abnormality, it immediately generates an exception record, including the room number, detection time, pre-set number of people, actual number of people detected, and the type of abnormality. These abnormality records are stored in the database, and the management system sends an early warning notification to the accommodation and reception facility management staff. This warning information can be displayed on the backend management interface or sent to management staff via SMS, app push, or email, allowing them to take timely countermeasures.

[0025] Through multi-source data fusion technology and intelligent anomaly detection algorithm, this invention provides a real-time, efficient and accurate method for identifying abnormal check-in status, effectively overcoming the limitations of traditional single monitoring methods, eliminating security management blind spots, and significantly improving the level of tourism industry safety management while protecting user privacy.

[0026] Step S2 specifically includes the following steps: S2.1. Millimeter-wave radar data collection: Deploy millimeter-wave radar equipment in public areas to collect data including the number of people, their location coordinates, and their movement trajectories. S2.2. Image Data Acquisition: Deploy image acquisition equipment to capture real-time image data of public areas. Using computer vision algorithms, extract data including the outlines and number of people and their behavioral characteristics. S2.3. Data time synchronization: Add timestamps to radar data and image data to ensure consistency in the time dimension.

[0027] In step S2.1, the millimeter-wave radar device transmits electromagnetic waves and receives reflected signals, and calculates the target person's movement trajectory, including distance and speed, based on time delay and Doppler frequency shift, where: The distance calculation formula is: Where d is the target distance, c is the speed of light, and t is the round-trip time difference of the signal; The speed calculation formula is: Where v is the target speed, f d is the frequency shift, and λ is the millimeter wave wavelength.

[0028] Specifically, to identify abnormal occupancy status in accommodation and reception venues, the present invention deploys millimeter-wave radar equipment and image acquisition equipment in step S2 to collect multi-dimensional data such as the number of people, location coordinates, and behavioral characteristics in public areas, and achieves spatiotemporal synchronization of this data through timestamps. The millimeter-wave radar equipment utilizes the reflection characteristics of electromagnetic waves to obtain the distance, speed, and movement trajectory of target individuals based on time delay and Doppler frequency shift. Simultaneously, the image acquisition equipment captures image data using computer vision algorithms to extract the outlines, number, and behavioral status of individuals, ultimately providing accurate data support for abnormal occupancy scenarios.

[0029] In this embodiment, millimeter-wave radar equipment is deployed in appropriate locations in public areas, covering the target area, to collect real-time information about people, including their number, location coordinates, and movement trajectories. Millimeter-wave radars transmit high-frequency electromagnetic wave signals and receive echo signals reflected by target people, extracting key information from them.

[0030] Millimeter-wave radar measures the distance between the target person and the device by the time delay of the electromagnetic wave signal. The distance calculation formula is as follows: Where: d represents the distance between the target and the millimeter-wave radar, c represents the speed of light, which is 3×108 m / s, and t represents the round-trip time difference between signal transmission and reception.

[0031] By measuring the time interval t between the transmitted signal and the echo signal, the real-time position coordinates of the target person can be obtained.

[0032] Millimeter-wave radar uses the Doppler effect to detect the frequency shift of electromagnetic waves and calculate the target person's movement speed. The speed calculation formula is as follows: Where: v is the target speed, f d is the frequency offset (Doppler shift), and λ is the wavelength of the millimeter wave (which can be calculated by frequency f=c / λ).

[0033] The detection of Doppler shift enables the radar to capture the dynamic trajectory of the target person, including the speed and direction of movement.

[0034] Through multiple continuous sampling, millimeter-wave radar can generate the target person's motion trajectory, combined with position coordinates and speed data, to provide support for subsequent abnormal behavior detection.

[0035] In this embodiment, image acquisition devices (such as cameras) are deployed in public areas, covering the main scenes of human activity, to capture image data in real time. The collected data is processed using computer vision algorithms to extract key information, including the outlines, number, and behavioral characteristics of people.

[0036] The image acquisition device captures real-time images at a set frame rate and identifies the number of people in the room through edge detection and target detection algorithms.

[0037] Use target detection algorithms (such as YOLO or SSD) to locate the bounding box of each person in the image, extract the outline information and count the number of people N 图像 .

[0038] At the same time, by removing background and noise, effective detection targets are retained, further improving detection accuracy.

[0039] Analyze the behavior of people in images using deep learning models (such as CNN), such as detecting smoking or other abnormal behaviors; Combined with the dynamic trajectory of people in the image, it helps determine whether there are any violations (such as gathering or abnormal movement).

[0040] In this embodiment, to ensure the consistency of millimeter-wave radar data and image data in the time dimension, the system adds a unified timestamp to the two types of data so that they can correspond to the physical state at the same moment and eliminate the time deviation between different data sources.

[0041] The system automatically generates a timestamp T each time it collects data. k , marking the time point of each frame of radar data and image data; the timestamp is provided by a high-precision clock to ensure millisecond-level synchronization accuracy.

[0042] Millimeter-wave radar data and image data are matched based on timestamps to ensure that the two data sources point to scene information at the same moment during data fusion and anomaly detection.

[0043] Step S2 achieves real-time detection of the number, location, and behavioral characteristics of individuals through the combined deployment of millimeter-wave radar and image acquisition equipment. The millimeter-wave radar calculates the distance and speed of individuals based on the time delay and Doppler shift of electromagnetic waves, providing dynamic trajectory data. The image acquisition equipment uses computer vision algorithms to extract the outlines, number, and behavioral characteristics of individuals, assisting in the identification of abnormal behavior. By adding timestamps, radar data and image data are synchronized in the temporal dimension, providing high-precision basic data support for subsequent data fusion and anomaly detection.

[0044] The image data processing in the preprocessing step in step S3 includes: Image enhancement, improving image quality through histogram equalization; Edge detection, using the Canny algorithm to extract the target contour; Background removal reduces irrelevant interference and retains the effective detection area.

[0045] The millimeter wave radar data preprocessing in step S3 includes: Gaussian filtering denoising to remove random noise; Mean smoothing to reduce errors; Timestamp synchronization to synchronize data with image data.

[0046] Specifically, in step S3 of the present invention, both image data and millimeter-wave radar data undergo preprocessing to improve data accuracy and quality, providing a reliable foundation for subsequent data fusion and anomaly detection. Image data preprocessing extracts effective information through image enhancement, edge detection, and background removal. Millimeter-wave radar data preprocessing ensures signal validity and consistency through denoising, smoothing, and timestamp synchronization. The specific preprocessing process is described in detail below.

[0047] In this embodiment, the preprocessing of image data mainly includes three steps: image enhancement, edge detection and background removal, which are as follows: Image enhancement optimizes the brightness distribution of the image through the histogram equalization method to improve image quality, enhance the contrast of the target area, and make the outline of people in the image clearer.

[0048] Implementation method: Histogram equalization redistributes the grayscale levels according to the pixel grayscale value distribution of the input image to make the pixel distribution more uniform.

[0049] For the input image I(x,y), the output image after histogram equalization is I ′ The (x,y) expression is: Where: CDF(I(x,y)) is the cumulative distribution function of the pixel grayscale value; M and N are the width and height of the image respectively; L is the grayscale level of the image.

[0050] The Canny algorithm is used to perform edge detection on the image and extract the contour information of the human target, thereby improving the accuracy of target detection in the image.

[0051] Implementation method: The implementation process of the Canny algorithm includes: Use Gaussian filtering to smooth the image and remove noise; Calculate the image gradient magnitude and direction; Remove non-edge points by non-maximum suppression; The double threshold method is used to extract strong edges and weak edges and perform edge connection.

[0052] Separate the foreground and background of the image, remove irrelevant interference information, and retain the effective area related to the detection target.

[0053] Implementation method: Use background modeling methods (such as those based on Gaussian mixture models (GMM)) to distinguish foreground from background through time series analysis: Where: K is the number of Gaussian distributions; w k is the weight of the k-th Gaussian distribution; η(x,μ k ,Σ k ) is the probability density function of the kth distribution, μ k is the mean, Σ k is the covariance matrix.

[0054] The background and foreground areas are distinguished through modeling, and irrelevant background is removed.

[0055] In this embodiment, the preprocessing of millimeter wave radar data includes Gaussian filtering denoising, mean smoothing, and timestamp synchronization, as follows: Since millimeter-wave radar signals may contain noise due to environmental interference, Gaussian filtering is needed to denoise the data to smooth the waveform and remove random interference.

[0056] Implementation method: The convolution kernel calculation formula of the Gaussian filter is: Where: σ is the standard deviation of the Gaussian distribution, which is used to control the range and intensity of the filter; x and y are the coordinate values ​​of the filter window.

[0057] The filter is convolved with the signal to obtain a smoothed signal.

[0058] Mean smoothing is used to eliminate error fluctuations in millimeter-wave radar data and further improve data stability.

[0059] Implementation method: Perform sliding window calculation on the sampling points: Where: S(i) is the current smoothing point; x k is the original data; n is the size of the sliding window.

[0060] Timestamps are added to millimeter-wave radar data and synchronized with image data, so that the two types of data remain consistent in the time dimension.

[0061] Implementation method: Each time data is collected, a high-precision timestamp T is generated. k and bind it with the corresponding radar signal; During subsequent processing, timestamp alignment is used to enable radar data and image data to point to scene information at the same moment.

[0062] In step S3 of the present invention, image data preprocessing uses histogram equalization to enhance image contrast, the Canny algorithm to extract target outlines, and background removal to reduce interference. Millimeter-wave radar data preprocessing uses Gaussian filtering for denoising and mean smoothing to reduce signal fluctuations, and timestamp synchronization to ensure spatiotemporal consistency with the image data. These preprocessing steps lay a solid foundation for subsequent data fusion and anomaly detection.

[0063] The multi-source fusion of millimeter-wave radar data and image data in step S4 is achieved through a weighted fusion algorithm. The fusion weight in the weighted fusion algorithm is dynamically adjusted according to the data confidence level. The calculation formula is: W 融合 =αW 雷达 +(1-α)W 图像 Among them, α is the weight of millimeter wave radar data, and 1-α is the weight of image data.

[0064] Specifically, in step S4 of the present invention, multi-source fusion of millimeter-wave radar data and image data is achieved using a weighted fusion algorithm. The goal of multi-source data fusion is to leverage the strengths of millimeter-wave radar and image acquisition equipment to fully extract effective information from both types of data, thereby improving data accuracy and robustness. To achieve this goal, the weighted fusion algorithm dynamically adjusts weights based on the confidence levels of the millimeter-wave radar and image data to optimize the fusion results.

[0065] In this embodiment, multi-source fusion adopts a weighted fusion algorithm to linearly combine millimeter-wave radar data and image data according to weights to generate a fused result.

[0066] The basic form of the weighted fusion algorithm is: W 融合 =αW 雷达 +(1-α)W 图像 in: W 融合 is the comprehensive result after fusion; W 雷达 is millimeter wave radar data; W 图像 is the image data; α is the weight of the millimeter-wave radar data, ranging from [0,1]; 1-α is the weight of the image data.

[0067] The value of the fusion weight α is dynamically adjusted according to the data confidence to ensure the accuracy of data fusion. The confidence is calculated as follows: Confidence C of millimeter wave radar data 雷达 : The confidence level of radar data is evaluated based on factors such as signal strength and noise level. The calculation formula is: Where: S 雷达 is the effective amplitude of the radar signal; N 雷达 is the noise amplitude of the radar signal.

[0068] Confidence C of image data 图像 : The confidence level of the target in the image is evaluated based on its clarity, edge strength and other indicators. The calculation formula is: Weight calculation formula: According to the confidence of radar data and image data, the fusion weight α is dynamically adjusted: 1-α is the weight of the image data.

[0069] The comprehensive data W obtained after fusion 融合 Includes the following: The real-time number of personnel; the spatial location coordinates of personnel; and the behavioral characteristics of personnel.

[0070] In this embodiment, the weighted fusion algorithm fully demonstrates the complementary advantages and disadvantages of millimeter-wave radar data and image data. Millimeter-wave radar data has advantages in detecting the number of people and their dynamic trajectories, while image data is more accurate in identifying people's behavioral characteristics and specific movements.

[0071] Abnormal state detection in step S5 includes: Number of people anomaly detection: the real-time number of people N detected 融合 With the preset number N 预设 Compare, when ΔN=N 融合 -N 预设 When >0, it is determined that the number of people is abnormal; Behavioral anomaly detection: Detect violations such as smoking or entering restricted areas using image data.

[0072] Specifically, in step S5 of the present invention, abnormal state detection analyzes the status of rooms and public areas based on the fused millimeter-wave radar and image data, combined with the accommodation venue's preset check-in rules. Abnormal states include abnormal headcount and abnormal behavior. Abnormal headcount is determined by comparing the actual headcount detected in real time with the preset headcount, while abnormal behavior is determined by analyzing image data to identify violations (such as smoking or entering restricted areas).

[0073] In this embodiment, the abnormal number of people is detected by the real-time number of people N 实时 With the preset number N 预设 If the actual number of people detected in the room exceeds the preset number specified in the accommodation and reception venue rules, the system will determine that the number of people is abnormal.

[0074] 1. Judgment conditions: The system detects abnormal number of people based on the following conditions: ΔN=N 实时 -N 预设 Where: ΔN is the difference in the number of people; N 实时 is the actual number of detected people after fusion; N 预设 The number of guests specified in the booking.

[0075] If ΔN>0, it is determined that the number of people is abnormal and the abnormal status record is triggered.

[0076] Implementation: The detection results N that combine millimeter wave radar data and image data 融合 , calculate the real-time number of people in the room; Compare the real-time number of people with the N stored in the database 预设 Compare and judge in real time whether the number of people exceeds the preset number; If an abnormality is established, the room number, time, number of people tested and other information with the abnormal number of people will be recorded.

[0077] Behavioral anomaly detection mainly uses image data to analyze people's movements and activity ranges to identify whether there is smoking behavior or illegal entry into restricted areas.

[0078] Smoking behavior detection: Objective: To detect smoking in public areas, especially in no-smoking areas.

[0079] Detection method: Smoke detection: Use deep learning models (such as CNN) to analyze whether there are smoke features in the image; The presence of smoke is determined based on the light scattering characteristics and local contrast changes in the image.

[0080] Motion Detection: Identify smoking actions through image object detection algorithms (such as YOLO); It mainly detects the approach behavior of hands and mouths and the characteristics of handheld objects.

[0081] Judgment conditions: When the smoke characteristics or smoking action confidence level detected exceeds the set threshold (such as confidence level P>0.8), it is determined that the smoking behavior is abnormal.

[0082] Restricted area entry detection: Objective: To detect whether people have entered the restricted area set by accommodation and reception venues.

[0083] Detection method: Obtain the real-time location coordinates of people through millimeter-wave radar data and image data; Compare the position coordinates to the preset restricted area, which is defined by the coordinate boundaries: (x,y)∈restricted area If a person's position falls within the restricted area, it will be considered as illegal entry into the restricted area.

[0084] Judgment condition: When the coordinate point (x, y) detected in real time meets the restricted area condition, an abnormal record is triggered.

[0085] Exception record generation: When the system detects an abnormal number of people or abnormal behavior, it generates an abnormal record, which includes: Abnormal type (abnormal number of people, smoking, entering restricted areas, etc.); Room number; detection time; comparison result of actual number of people with preset number of people.

[0086] Abnormal status feedback: The system will provide real-time feedback of abnormal status to the accommodation reception venue manager through SMS, APP notification or background system reminder.

[0087] The exception record in step S6 includes the exception type, room number, detection time, number of people in the order, and the actual number of people detected.

[0088] Specifically, in step S6 of the present invention, when an abnormal condition is detected, the system generates an abnormality record and sends an early warning notification to the accommodation and reception facility management via the management system. The abnormality record, as key event data, contains specific information about the abnormal condition, facilitating rapid response and subsequent data analysis by management personnel.

[0089] In this embodiment, the system will immediately generate an exception record after detecting an abnormal state. The exception record contains the following key contents: Exception Type: Describe the kind of exception that occurred, for example: Abnormal number of people (e.g. check-in exceeding the number of people in the reservation); Smoking behavior (especially detection of smoking movements or smoke characteristics in no-smoking areas); Restricted area entry (personnel entering an unauthorized restricted area).

[0090] Room Number: Record the specific room number where the exception occurred to facilitate management personnel to locate the problem room.

[0091] Detection time: Use a high-precision timestamp to record the time when the abnormal state occurs, such as "2024-12-18 14:35:10", to ensure the exact time point of the abnormal event.

[0092] Number of orders: Extract the preset number of people (N) registered in the order from the accommodation reception system database 预设 ) for comparison with the actual number of people tested.

[0093] Actual number of people tested: The real-time number of people detected (N) obtained by analyzing the fused millimeter-wave radar and image data 实时 ).

[0094] When the actual number of people tested exceeds the preset number, the system will mark the exception as overcrowding.

[0095] Example of generating a record: When an abnormal state of overcrowding in a room is detected, an example of abnormal record content is as follows: Abnormal type: Abnormal number of people Room number: 302 Detection time: 2024-12-18 14:35:10 Number of orders: 2 Actual number of people tested: 4 Storing exception records: The system stores exception records in the background database for accommodation and reception venue managers to query and analyze at any time.

[0096] Data structured storage example: Long-term data storage: Archive historical abnormality records to provide data support for subsequent statistical analysis and optimization of safety management strategies for accommodation and reception venues.

[0097] In this embodiment, when the system detects an anomaly, it will trigger an early warning notification based on the anomaly type and send relevant information to the management personnel in a timely manner. Warning method: Backend management system: Abnormal status will be prompted in the management system interface in the form of pop-up windows or warning messages.

[0098] SMS notification: The system sends abnormal information to the mobile phone of the accommodation reception venue manager through the SMS interface.

[0099] APP push: If the accommodation reception venue uses a dedicated management APP, the system will push real-time notifications to relevant personnel.

[0100] Email: The system will generate an email for the exception record and send it to the designated management mailbox.

[0101] Warning notice content: The specific information contained in the early warning notice is as follows: Abnormal type (e.g., unusual number of people, smoking behavior, entry into restricted areas); Room number or area name; Detection time; The number of people in the order and the actual number of people tested (for abnormal numbers); Description of the illegal action (smoking or entering a restricted area).

[0102] Example of warning information: If an overcrowding is detected, the following warning information will be sent: Backend management system: Warning! Room number: 302, detected as overcrowded. Number of people ordered: 2, actual number of people checked: 4. Detection time: 2024-12-18 14:35:10. Please address promptly.

[0103] SMS notification: Accommodation and reception venue warning: Room 302 is abnormally overcrowded. The number of guests booked is 2, but the actual number of guests is 4. Time: 2024-12-18 14:35:10. Please verify promptly.

[0104] In this embodiment, the abnormality record and the early warning notification are linked, that is, when the abnormality is detected and the record is generated, the system automatically triggers the early warning notification function. In this way, it ensures that the management personnel can be informed of the abnormal status in time and take appropriate measures.

[0105] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for identifying abnormal occupancy status by integrating radar and image data, characterized in that: The following steps are involved: S1. Obtain check-in order information and configure check-in rules, which include room number, preset number of people, non-smoking area and restricted area settings; S2. Collect personnel data in real time through millimeter wave radar equipment and image acquisition equipment, including the number, location and behavior status of personnel; S3, preprocessing the collected millimeter wave radar data and image data, wherein the preprocessing includes denoising, standardization and time synchronization processing; S4, performing multi-source fusion on the pre-processed millimeter-wave radar data and image data, wherein the data fusion includes weighted fusion and Kalman filtering processing; S5. Detect abnormal conditions based on the data fusion results and preset rules, where the abnormal conditions include overcrowding, illegal behavior in no-smoking areas, and entering restricted areas; S6. Generate an abnormal record and send an early warning notification to the management personnel through the management system.

2. The abnormal occupancy status recognition method of fusing radar and image data according to claim 1 is characterized in that: The S2 step specifically includes the following steps: S2.

1. Millimeter-wave radar data collection: Deploy millimeter-wave radar equipment in public areas to collect data including the number of people, their location coordinates, and their movement trajectories; S2.2, Image data acquisition: deploy image acquisition equipment to capture image data of public areas in real time, and extract data including the outline and number of people and their behavioral characteristics through computer vision algorithms; S2.3, Data time synchronization: Add timestamps to radar data and image data to make them consistent in the time dimension.

3. The abnormal occupancy status identification method of fusing radar and image data according to claim 2 is characterized in that: In step S2.1, the millimeter wave radar device transmits electromagnetic waves and receives reflected signals, and calculates the target person's moving trajectory, including distance and speed, based on time delay and Doppler frequency shift, wherein: The distance calculation formula is: Where d is the target distance, c is the speed of light, and t is the round-trip time difference of the signal; The speed calculation formula is: Where v is the target speed, f d is the frequency shift, and λ is the wavelength of the millimeter wave.

4. The abnormal occupancy status recognition method of fusing radar and image data according to claim 1, characterized in that: The multi-source fusion of the millimeter-wave radar data and the image data in step S4 is realized by a weighted fusion algorithm. The fusion weight in the weighted fusion algorithm is dynamically adjusted according to the data confidence. The calculation formula is: W 融合 =αW 雷达 +(1-α)W 图像 Among them, α is the weight of millimeter wave radar data, and 1-α is the weight of image data.

5. The abnormal occupancy status identification method of fusing radar and image data according to claim 1 is characterized in that: The abnormal state detection in step S5 includes: Abnormal number detection: The number of people detected in real time N 融合 With the preset number of people N 预设 Compare, when ΔN=N 融合 -N 预设 When >0, it is judged as abnormal number of people; Behavioral anomaly detection: Use image data to detect violations such as smoking or entering restricted areas.

6. The abnormal occupancy status identification method of fusing radar and image data according to claim 1, characterized in that: The image data processing in the preprocessing step in step S3 includes: Image enhancement, improving image quality through histogram equalization; Edge detection, using the Canny algorithm to extract the target contour; Background removal reduces irrelevant interference and retains the effective detection area.

7. The abnormal occupancy status identification method of fusing radar and image data according to claim 1 is characterized in that: The abnormal record in step S6 includes the abnormal type, room number, detection time, number of people in the order and the actual number of people detected.

8. The abnormal occupancy status identification method of fusing radar and image data according to claim 1, characterized in that: The millimeter wave radar data preprocessing in step S3 includes: Gaussian filtering denoising, removes random noise; Mean smoothing to reduce errors; Timestamp synchronization to synchronize data with image data.

Images