A multi-terminal self-bootstrapping deployment optimization method and related apparatus

Abstract

This invention discloses a multi-terminal self-guided deployment optimization method and related apparatus. The method includes: collecting multi-view images and timestamp data of a living environment, preprocessing them to obtain a dataset; constructing a structured box scene model based on the dataset, representing the worst-case acoustic propagation conditions; defining functional areas and sampling points where abnormal events may occur, as well as feasible candidate terminal installation points under physical constraints, in the structured box scene model, and constructing an observability matrix that comprehensively characterizes the ability of any candidate terminal installation point to perceive abnormal events at any location in space; and searching for the optimal terminal deployment combination among all candidate installation points based on the observability matrix, with the number of terminals as a constraint. This invention frees the terminal deployment process from dependence on human experience, making it computationally achievable, verifiable, and globally optimizable.

Description

Technical Field

[0001] This invention belongs to the field of intelligent sensing technology, and specifically relates to a multi-terminal self-guided deployment optimization method and related devices. Background Technology

[0002] With the development of smart home and elderly care monitoring technologies, abnormal state monitoring systems based on multimodal perception, such as sound and vision, are increasingly widely used in areas such as care for elderly people living alone. While visual monitoring is direct and effective, its inherent privacy intrusion often leads to low user acceptance. In contrast, monitoring solutions that identify specific abnormal sounds such as falls or cries for help demonstrate significant advantages in protecting privacy, making them a reasonable and important technological choice.

[0003] However, existing sound-based monitoring solutions generally suffer from a key flaw: most focus on optimizing backend recognition algorithms, while the deployment of frontend monitoring terminals heavily relies on the personal experience of implementers, lacking systematic and quantifiable modeling and optimization guidance. In real-world home environments, complex spatial structures, large furniture obstructing the view, and the acoustic reflection and shielding effects of various material interfaces (such as glass and metal) significantly alter sound propagation characteristics, severely impacting the coverage reliability, signal quality, and collaborative sensing accuracy of distributed monitoring networks. This haphazard deployment directly leads to problems such as unstable overall system performance, numerous monitoring blind spots, significant environmental interference, and high false alarm rates. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-terminal self-guided deployment optimization method and related device to solve the problems of unstable performance, many monitoring blind spots, large environmental interference, and high false alarm rate in the existing technology.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a multi-terminal self-booting deployment optimization method, comprising: Collect multi-view images and timestamp data of the living environment, and obtain the dataset after preprocessing; Based on the dataset, a structured box scene model representing the worst-case acoustic propagation conditions is constructed; In the structured box scene model, functional areas and sampling points where abnormal events may occur are defined, as well as candidate terminal installation points that are feasible under physical constraints. An observability matrix is ​​constructed to comprehensively characterize the ability of any candidate terminal installation point to perceive abnormal events at any location in space. Based on the observability matrix, and with the number of terminals as a constraint, the optimal combination of terminal deployments is searched among all candidate installation points.

[0006] Furthermore, the collected multi-view images and timestamp data of the living environment, after preprocessing, yield a dataset including: The system uses a mobile device to systematically capture images of the living environment, simultaneously collecting multi-view images and timestamp information. It performs localized privacy filtering, extracts keyframes and adjusts the resolution of the images, and constructs a minimal dataset for 3D reconstruction.

[0007] Furthermore, the localized privacy filtering process includes blurring or cropping the face region, human body contour region, or preset sensitive region in the collected data, and performing keyframe extraction or resolution downsampling on the collected data.

[0008] Furthermore, the construction of a structured box scene model representing the worst-case acoustic propagation conditions based on the dataset includes: Based on the dataset, a 3D point cloud model of the room is generated. Through semantic segmentation and spatial clustering, the main furniture and partition objects are abstracted into 3D bounding boxes. Each bounding box is uniformly assigned preset parameters that represent high acoustic occlusion and reflection characteristics, and a structured box scene model representing the worst-case acoustic propagation conditions is constructed.

[0009] Furthermore, based on the dataset, a 3D reconstruction process is performed on a cloud computing platform to generate a 3D point cloud model or planar structure model of the living environment, and the point cloud data is subjected to scale normalization and coordinate alignment processing under a unified spatial coordinate system. The acquired multi-view images are subjected to target detection and semantic classification processing to obtain the relationship between object category and spatial projection, and the semantic classification results are mapped to point cloud space. Based on the rules of spatial continuity and semantic consistency, spatial grouping processing is performed on the point cloud data, and a corresponding three-dimensional bounding box is constructed for each spatial group. The bounding box is in the form of an axis-aligned bounding box or a directed bounding box. Based on box volume threshold, spatial adjacency relationship and semantic consistency constraints, automatic filtering and merging processing is performed on adjacent or overlapping 3D bounding boxes to generate a set of boxes with consistent structure, which is used to characterize potential occlusion units in the living space. The set of bounding boxes is uniformly assigned preset acoustic impact attribute parameters, which include at least occlusion weight parameters and reflection risk weight parameters. The three-dimensional bounding box set and its corresponding acoustic impact attribute parameters are combined to construct a structured box scene model. The structured box scene model is used for subsequent abnormal event propagation path construction, observability matrix generation, and terminal deployment optimization.

[0010] Furthermore, in the structured box scene model, functional areas and sampling points where abnormal events may occur are defined, as well as feasible candidate terminal installation points under physical constraints; an observability matrix is ​​constructed to comprehensively characterize the ability of any candidate terminal installation point to perceive abnormal events at any location in space, including: In the structured box scene model, functional areas and sampling points where abnormal events may occur are defined, as well as feasible candidate terminal installation points under physical constraints. By calculating the direct propagation path between each event point and each installation point, and analyzing its intersection relationship with each bounding box, the equivalent occlusion degree and signal attenuation of the path are quantified, and an observability matrix that comprehensively characterizes the ability of any candidate point to perceive abnormal events at any location in space is generated.

[0011] Based on the structured box scene model, the living space is divided into functional areas, and an abnormal event sampling point set is generated within the functional areas. The abnormal event sampling points are used to characterize the possible location distribution of abnormal events in the space. A set of candidate terminal deployment points is generated in the space, and the candidate terminal deployment points are filtered based on physical placement constraints; the physical placement constraints include at least placement plane constraints, preset height range constraints, and minimum safe distance constraints from noise sources. A spatial propagation path is constructed for each abnormal event sampling point and each candidate terminal deployment point; Based on propagation distance, occlusion counting model and reflection risk weight parameters, the perception capability of each abnormal event sampling point and candidate terminal deployment point is quantitatively evaluated, and an abnormal event observability matrix is ​​generated to characterize the perception distribution relationship of abnormal events to terminal deployment locations in space. A multi-terminal collaborative structure constraint model is constructed based on the observability matrix of the abnormal events. The collaborative structure constraint model is used to constrain the coverage complementarity relationship and redundancy suppression relationship among multiple terminals.

[0012] Furthermore, the step of searching for the optimal terminal deployment combination among all candidate installation points based on the observability matrix and constrained by the number of terminals includes: Based on the observability matrix, with the number of terminals as a constraint, the optimal terminal deployment combination is searched among all candidate installation points. The search aims to maximize the overall observability of the system for all abnormal event sampling points, and takes the coverage complementarity and redundancy suppression between terminals as cooperative constraints. By solving this optimization problem, the optimal deployment scheme is output. Subsequently, by introducing perturbations into the model parameters, the robustness of the scheme under environmental parameter fluctuations is verified. Based on the set of candidate terminal deployment points and the constraint of the number of terminals, multiple candidate terminal deployment combinations are generated to construct a system-level deployment scheme space.

[0013] Based on the observability matrix of abnormal events and the multi-terminal collaborative structure constraint model, the system-level perception capability of the candidate terminal deployment combinations is scored, and the candidate terminal deployment combinations are ranked according to the scoring results. The terminal deployment combination with the best score is selected as the optimal deployment scheme, and the abnormal event perception structure is reverse-analyzed based on the optimal deployment scheme to verify the consistency relationship between the deployment structure and the spatial distribution of abnormal events. The optimal deployment scheme is subjected to perturbation verification processing. By applying perturbations within a preset range to the occlusion weight parameter, reflection risk weight parameter and cooperative structure constraint parameter, the stability of the deployment scheme under environmental change conditions is evaluated.

[0014] Secondly, the present invention provides a multi-terminal self-booting deployment optimization system, comprising: The data acquisition module is used to collect multi-view images and timestamp data of the living environment, and obtain the dataset after preprocessing; The model building module is used to build a structured box scene model representing the worst-case acoustic propagation conditions based on the dataset. The perception module is used to define functional areas and sampling points where abnormal events may occur in the structured box scene model, as well as feasible candidate terminal installation points under physical constraints, and to construct an observability matrix that comprehensively describes the ability of any candidate terminal installation point to perceive abnormal events at any location in space. The output module is used to search for the optimal combination of terminal deployments among all candidate installation points based on the observability matrix and with the number of terminals as a constraint.

[0015] Thirdly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the multi-terminal self-booting deployment optimization method.

[0016] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the multi-terminal self-booting deployment optimization method.

[0017] Compared with the prior art, the present invention has the following technical effects: This invention, through systematic modeling and closed-loop optimization, transforms the previously experience-dependent terminal deployment into a computable, guided, and adaptive intelligent engineering process, specifically manifested in: This invention, for the first time, constructs the multi-terminal deployment problem as a reverse optimization problem based on a spatial acoustic model and system-level perception performance objectives. By using an observability matrix and cooperative constraints for quantitative solution, the deployment process is freed from absolute dependence on human experience, becoming computationally achievable, verifiable, and globally optimizable.

[0018] By abstracting complex environments into "structured box scene models" with acoustic properties, the computational complexity of 3D reconstruction and acoustic simulation is significantly reduced. Furthermore, the "worst-case" unified assignment strategy avoids overfitting, enabling the generated deployment scheme to have stronger environmental adaptability and anti-interference capabilities.

[0019] System-level optimization based on observability matrix and multi-terminal collaborative constraint model can proactively minimize coverage blind spots and rationalize perception redundancy, thereby simultaneously improving the detection rate and recognition accuracy of abnormal events in complex home environments and effectively reducing false alarms and missed alarms.

[0020] The entire process, guided by event-triggered graphical interface, significantly reduces the user's workload. After deployment, the system can automatically calibrate the model through multi-terminal joint verification and trigger re-optimization when the environment changes, forming a complete closed loop of "guided deployment - actual verification - adaptive optimization," ensuring the long-term effectiveness of the system.

[0021] All raw data undergoes privacy filtering on the device side, with only minimal modeling data uploaded, fundamentally preventing privacy leaks. Adopting an architecture of "complex modeling / optimization in the cloud + lightweight execution on the device side," users can obtain optimal deployment solutions without specialized equipment or knowledge. Furthermore, the cloud-based algorithm is continuously updated and iterated, providing users with long-term evolving technical service capabilities. Attached Figure Description

[0022] Figure 1 This is a block diagram of the overall system structure of the present invention; Figure 2 A schematic diagram of a structured acoustic box scenario; Figure 3 This is a schematic diagram of the observability matrix for anomalous events; Figure 4 This diagram illustrates the deployment optimization and system adaptive closed-loop design. Detailed Implementation

[0023] The present invention will be further described below with reference to the accompanying drawings: Example 1, please refer to Figure 1 This invention provides a multi-terminal self-booting deployment optimization method, comprising: Collect multi-view images and timestamp data of the living environment, and obtain the dataset after preprocessing; Based on the dataset, a structured box scene model representing the worst-case acoustic propagation conditions is constructed; In the structured box scene model, functional areas and sampling points where abnormal events may occur are defined, as well as candidate terminal installation points that are feasible under physical constraints. An observability matrix is ​​constructed to comprehensively characterize the ability of any candidate terminal installation point to perceive abnormal events at any location in space. Based on the observability matrix, and with the number of terminals as a constraint, the optimal combination of terminal deployments is searched among all candidate installation points.

[0024] This invention constructs a three-dimensional spatial structure model by integrating visual information from the environment and establishes a quantitative mapping relationship between spatial physical constraints and the observability of abnormal events by combining acoustic propagation laws. Based on this, the empirical problem of "how to deploy terminals" is transformed into a reverse-engineering and optimization problem of "how to meet the overall perception performance requirements of the system," thereby generating a visualized optimal deployment scheme. Ultimately, this allows ordinary users to quickly complete the installation of a highly reliable monitoring network without professional acoustic knowledge, guided by the system, and enables long-term adaptive adjustments based on environmental changes. This fundamentally protects privacy and security while improving the coverage and accuracy of abnormal state identification.

[0025] Example 2, please refer to Figures 1 to 4 This invention provides a multi-terminal self-booting deployment optimization method, specifically including: S1: Privacy-controlled environment for data collection and preprocessing Users are guided to use mobile devices to systematically photograph their living environment, simultaneously collecting multi-view images and timestamp information; before data upload, localized privacy filtering is performed, and keyframe extraction and resolution adjustment are performed on the images to construct a minimal dataset for 3D reconstruction; after encryption, the dataset is uploaded to the cloud computing platform.

[0026] S1.1 Acquisition Mode Initialization and Path Guidance When the user terminal enters the deployment guidance mode, the system initializes the spatial acquisition mode and generates spatial acquisition guidance information in the user interface to guide the user to take pictures and collect data along the boundaries of the room, main passages and functional areas.

[0027] The guidance information includes shooting height range prompts, shooting angle prompts, and spatial coverage progress prompts to ensure that the acquisition path can cover the structural boundaries of the room and major obstructed areas.

[0028] S1.2 Multi-source data synchronous acquisition The system acquires multi-view image data and timestamp information simultaneously during the acquisition process.

[0029] S1.3 Acquisition Quality Assessment and Supplementary Acquisition Mechanism The system performs real-time quality assessment on the collected data, which includes at least image sharpness assessment, spatial coverage integrity assessment, and viewpoint continuity assessment.

[0030] When the quality assessment result is lower than the preset threshold, the system outputs a supplementary sampling prompt to the user to guide the user to re-collect data in the missing area.

[0031] S1.4 Privacy Preprocessing and Minimized Data Construction Before uploading data, the system performs privacy protection processing on the collected images or videos, including at least blurring or cropping the face area, body outline area, and preset sensitive areas.

[0032] In a preferred embodiment, the system further performs keyframe extraction or resolution downsampling on the acquired data to construct only the minimum necessary dataset for 3D reconstruction.

[0033] S1.5 Data Encryption and Upload The system encrypts the minimum necessary dataset and uploads the data to the cloud computing platform through a fragmented upload mechanism for subsequent spatial modeling and deployment optimization.

[0034] S2: Structured Spatial Abstraction for Acoustic Occlusion Modeling Based on the uploaded dataset, a 3D point cloud model of the room is generated in the cloud. Through semantic segmentation and spatial clustering, the main furniture, partitions and other objects are abstracted into a series of 3D bounding boxes. Each bounding box is uniformly assigned preset parameters that represent high acoustic occlusion and reflection characteristics, thereby constructing a "structured box scene model" that represents the worst-case acoustic propagation conditions.

[0035] S2.1 3D Point Cloud Generation and Spatial Coordinate Unification Based on the original dataset of the spatial structure, the system performs three-dimensional reconstruction processing on the cloud computing platform to generate a sparse point cloud model or planar structure model of the living environment, and performs scale normalization and coordinate alignment processing on the point cloud data under a unified spatial coordinate system.

[0036] S2.2 Point Cloud Semantic Mapping and Initial Construction of 3D Box The system performs target detection and semantic classification processing on the acquired multi-view images, obtains the relationship between object categories and spatial projection, and maps the semantic classification results to point cloud space.

[0037] Based on spatial continuity and semantic consistency rules, the system performs spatial grouping processing on point cloud data and constructs a corresponding 3D bounding box for each spatial group. The bounding box can be an axis-aligned bounding box or a directed bounding box.

[0038] S2.3 Box Selection and Structural Consistency Integration Based on box volume thresholds, spatial adjacency relationships, and semantic consistency constraints, the system automatically filters and merges adjacent or overlapping 3D bounding boxes to generate a set of boxes with consistent structures, which is used to characterize potential occlusion units in living spaces.

[0039] S2.4 Worst-case acoustic property uniform mapping The system uniformly assigns preset acoustic impact attribute parameters to the set of boxes, and the acoustic impact attribute parameters include at least occlusion weight parameters and reflection risk weight parameters.

[0040] In a preferred embodiment, the system assigns parameter values ​​to all types of objects and furniture according to the worst-case acoustic model with high reflectivity and high occlusion, in order to construct a spatial abstract model under the most unfavorable conditions for the ability to perceive abnormal events.

[0041] S2.5 Structured Box Scene Model Generation The system constructs a structured box scene model by combining the three-dimensional bounding box set and its corresponding acoustic influence attribute parameters. The structured box scene model is used for subsequent abnormal event propagation path construction, observability matrix generation, and terminal deployment optimization.

[0042] S3: Quantitative Modeling of Observability of Anomalies In the structured box scene model, functional areas and sampling points where abnormal events (such as falls) may occur are defined, as well as candidate terminal installation points that are feasible under physical constraints. By calculating the direct propagation path between each event point and each installation point, and analyzing its intersection with each bounding box, the equivalent occlusion degree and signal attenuation of the path are quantified, and finally an observability matrix that comprehensively describes the ability of any candidate point to perceive abnormal events at any location in space is generated.

[0043] S3.1 Functional Area Identification and Abnormal Event Sampling Point Generation Based on the structured box scene model, the system divides the living space into functional areas and generates a set of abnormal event sampling points within the functional areas. The abnormal event sampling points are used to characterize the possible location distribution of abnormal events in the space.

[0044] S3.2 Candidate Terminal Deployment Point Construction and Physical Constraint Filtering The system generates a set of candidate terminal deployment points in the space and filters the candidate terminal deployment points based on physical placement constraints.

[0045] The physical placement constraints include at least the placeable plane constraint, the preset height range constraint, and the minimum safe distance constraint from the noise source.

[0046] S3.3 Propagation Path Construction and Occlusion Counting Model The system constructs a spatial propagation path for each abnormal event sampling point and each candidate terminal deployment point.

[0047] In a preferred embodiment, the propagation path is a spatial connection between the abnormal event sampling point and the candidate terminal deployment point, and the number of crossings and intersections of the connection with the boxes in the structured box scene model are counted to construct an occlusion counting model.

[0048] S3.4 Observability Quantization and Matrix Generation Based on propagation distance, occlusion counting model and reflection risk weight parameters, the system quantifies and evaluates the perception capability of each abnormal event sampling point and candidate terminal deployment point, and generates an abnormal event observability matrix to characterize the perception distribution relationship of abnormal events on terminal deployment locations in space.

[0049] S3.5 Multi-terminal Collaborative Structure Constraint Construction The system constructs a multi-terminal collaborative structure constraint model based on the abnormal event observability matrix. The collaborative structure constraint model is used to constrain the coverage complementarity relationship and redundancy suppression relationship among multiple terminals.

[0050] S4: Reverse engineering and robustness verification of system-level deployment solutions Based on the observability matrix, with the number of terminals as a constraint, the optimal terminal deployment combination is searched among all candidate installation points. The search aims to maximize the overall observability of the system for all abnormal event sampling points, and takes the coverage complementarity and redundancy suppression between terminals as cooperative constraints. By solving this optimization problem, the optimal deployment scheme is output. Subsequently, by introducing perturbations into the model parameters, the robustness of the scheme under environmental parameter fluctuations is verified.

[0051] S4.1 Deployment Combination Generation Based on the set of candidate terminal deployment points and the constraint of the number of terminals, the system generates multiple candidate terminal deployment combinations to construct a system-level deployment scheme space.

[0052] S4.2 System-level observability scoring and ranking The system scores the system-level perception capability of the candidate terminal deployment combinations based on the anomaly event observability matrix and the multi-terminal collaborative structure constraint model, and sorts the candidate terminal deployment combinations according to the scoring results.

[0053] S4.3 Optimal Deployment Scheme Selection and Backward Solution The system selects the terminal deployment combination with the best score as the optimal deployment scheme, and performs reverse analysis on the abnormal event perception structure based on the optimal deployment scheme to verify the consistency relationship between the deployment structure and the spatial distribution of abnormal events.

[0054] S4.4 Stability Assessment and Disturbance Verification The system performs perturbation verification processing on the optimal deployment scheme. By applying perturbations within a preset range to the occlusion weight parameter, reflection risk weight parameter, and cooperative structure constraint parameter, the stability of the deployment scheme under environmental change conditions is evaluated.

[0055] S5: Multi-terminal joint verification and long-term adaptive optimization After the user installs the terminal according to the plan, the system collects the actual response data of each terminal by triggering standardized simulated events (such as playing a specific test tone) and builds a measured perception model. By comparing the measured model with the predicted observability matrix, the acoustic model parameters are adaptively calibrated. If the deviation is too large, re-optimization is triggered and the user is guided to adjust the terminal position, thereby realizing the self-perception, self-verification and self-optimization closed loop of the system in long-term operation.

[0056] S5.1 Joint Verification Event Construction After the system is deployed on the terminal, it generates a standardized set of joint verification events to simulate the propagation process of abnormal events in space.

[0057] S5.2 Measured Sensing Structure Acquisition The system collects perception response data from multiple terminals to the joint verification event and constructs a measured abnormal event perception structure model.

[0058] S5.3 Consistency Assessment and Parameter Correction The system performs a consistency comparison between the measured abnormal event perception structure model and the predicted abnormal event observability matrix, and adaptively adjusts the occlusion weight parameters, reflection risk weight parameters, and cooperative structure constraint parameters based on the comparison results.

[0059] S5.4 Deployment Structure Re-optimization Trigger When the consistency assessment result is lower than the preset consistency threshold, the system triggers the terminal deployment scheme re-optimization process and outputs terminal location adjustment guidance information to the user.

[0060] S5.5 Long-term operation and environmental change monitoring During long-term operation, the system continuously monitors changes in terminal location and spatial structure. When a change is detected that exceeds a preset threshold, it automatically triggers a re-verification and deployment optimization process.

[0061] Example 3: This invention provides a multi-terminal self-booting deployment optimization method, comprising: Users initiate the system's deployment guide mode via smartphones or other mobile devices, and follow the on-screen prompts to move along room boundaries and main functional areas such as the living room, bedroom, and bathroom to acquire images from multiple perspectives. During the acquisition process, the system automatically performs localized blurring processing on sensitive information such as faces and human bodies in the images, constructing a raw dataset of spatial structures for modeling.

[0062] After the dataset is uploaded to the cloud, the system executes a 3D reconstruction algorithm to generate a 3D point cloud model of the environment, and then extracts the main planar structures such as walls and floors, and performs 3D bounding box fitting on the clustered furniture point cloud.

[0063] Next, the system invokes an object detection algorithm to obtain two-dimensional semantic information and maps the semantic labels to corresponding three-dimensional bounding boxes through multi-view geometric relationships. If the semantic confidence is insufficient, the system will proactively prompt the user for simple confirmation. Then, based on the semantic category (such as "sofa" or "glass cabinet"), each bounding box is uniformly assigned preset simplified acoustic attribute parameters (such as occlusion coefficient and reflection coefficient), thereby constructing a structured box scene model for acoustic analysis.

[0064] Based on this model, the system automatically generates a series of abnormal event sampling points in high-risk areas such as around the bed and in the bathroom, while simultaneously generating candidate terminal deployment points in locations that meet physical installation constraints, such as walls and ceilings. By calculating the propagation path of each "event-deployment point" pair and its interaction with the scene model, the system quantifies and generates an abnormal event observability matrix. Based on this, with the deployment of two terminals as a constraint, the system uses an optimization algorithm to perform a reverse solution, outputting the optimal deployment coordinates with the most comprehensive coverage and the most reasonable redundancy.

[0065] After the user completes the terminal installation as shown in the diagram, the system can initiate the joint verification mode: playing a standardized test audio source, collecting the actual response signals of each terminal, and constructing a measured observability model. This measured model is then compared with the predicted model. If significant deviations exist, the system can automatically correct the acoustic propagation parameters and, if necessary, generate location adjustment suggestions, achieving continuous calibration and long-term adaptive optimization after deployment.

[0066] In a preferred embodiment, the room environment modeling process is a one-time or low-frequency execution process, and the overall modeling time is within a preset short window range.

[0067] Before entering the modeling and acquisition mode, the system outputs a personnel avoidance prompt message to the user terminal, guiding family members to temporarily avoid the acquisition area within the short window, thereby further reducing the risk of personal identity information or privacy content being collected without affecting the quality of 3D modeling.

[0068] After completing the room environment modeling, the system enters a long-term operation mode, performing abnormal state monitoring, deployment verification, and adaptive optimization processes solely based on the structured acoustic box scene model and terminal deployment strategy. This eliminates the need to repeatedly collect room environment images or video data, thereby minimizing the frequency of data collection involving privacy information throughout the system's entire lifecycle.

[0069] In another embodiment of the present invention, a multi-terminal self-booting deployment optimization system is provided, which can be used to implement the above-mentioned multi-terminal self-booting deployment optimization method. Specifically, the system includes: The data acquisition module is used to collect multi-view images and timestamp data of the living environment, and obtain the dataset after preprocessing; The model building module is used to build a structured box scene model representing the worst-case acoustic propagation conditions based on the dataset. The perception module is used to define functional areas and sampling points where abnormal events may occur in the structured box scene model, as well as feasible candidate terminal installation points under physical constraints, and to construct an observability matrix that comprehensively describes the ability of any candidate terminal installation point to perceive abnormal events at any location in space. The output module is used to search for the optimal combination of terminal deployments among all candidate installation points based on the observability matrix and with the number of terminals as a constraint.

[0070] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0071] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used in the operation of a multi-terminal self-bootstrapping deployment optimization method.

[0072] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the multi-terminal self-booting deployment optimization method in the above embodiments.

[0073] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0074] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0075] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0076] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A multi-terminal self-booting deployment optimization method, characterized in that, include: Collect multi-view images and timestamp data of the living environment, and obtain the dataset after preprocessing; Based on the dataset, a structured box scene model representing the worst-case acoustic propagation conditions is constructed; In the structured box scene model, functional areas and sampling points where abnormal events may occur are defined, as well as candidate terminal installation points that are feasible under physical constraints. An observability matrix is ​​constructed to comprehensively characterize the ability of any candidate terminal installation point to perceive abnormal events at any location in space. Based on the observability matrix, and with the number of terminals as a constraint, the optimal combination of terminal deployments is searched among all candidate installation points.

2. The multi-terminal self-booting deployment optimization method according to claim 1, characterized in that, The collected multi-view images and timestamp data of the living environment are preprocessed to obtain a dataset, including: The system uses a mobile device to systematically capture images of the living environment, simultaneously collecting multi-view images and timestamp information. It performs localized privacy filtering, extracts keyframes and adjusts the resolution of the images, and constructs a minimal dataset for 3D reconstruction.

3. The multi-terminal self-booting deployment optimization method according to claim 2, characterized in that, The localized privacy filtering process includes blurring or cropping the face region, human body contour region, or preset sensitive region in the collected data, and performing keyframe extraction or resolution downsampling on the collected data.

4. The multi-terminal self-booting deployment optimization method according to claim 1, characterized in that, The construction of a structured box scene model representing the worst-case acoustic propagation conditions based on the dataset includes: Based on the dataset, a 3D point cloud model of the room is generated. Through semantic segmentation and spatial clustering, the main furniture and partition objects are abstracted into 3D bounding boxes. Each bounding box is uniformly assigned preset parameters that represent high acoustic occlusion and reflection characteristics, and a structured box scene model representing the worst-case acoustic propagation conditions is constructed.

5. The multi-terminal self-booting deployment optimization method according to claim 4, characterized in that, Based on the dataset, a 3D reconstruction process is performed on a cloud computing platform to generate a 3D point cloud model or planar structure model of the living environment, and the point cloud data is subjected to scale normalization and coordinate alignment processing under a unified spatial coordinate system. Target detection and semantic classification processing are performed on the acquired multi-view images to obtain the relationship between object categories and spatial projection, and the semantic classification results are mapped to point cloud space; Based on spatial continuity and semantic consistency rules, spatial grouping processing is performed on point cloud data, and a corresponding three-dimensional bounding box is constructed for each spatial group. The bounding box is in the form of an axis-aligned bounding box or a directed bounding box. Based on box volume threshold, spatial adjacency relationship and semantic consistency constraints, automatic filtering and merging processing is performed on adjacent or overlapping 3D bounding boxes to generate a set of boxes with consistent structure, which is used to characterize potential occlusion units in the living space. The set of bounding boxes is uniformly assigned preset acoustic impact attribute parameters, which include at least occlusion weight parameters and reflection risk weight parameters. The three-dimensional bounding box set and its corresponding acoustic impact attribute parameters are combined to construct a structured box scene model. The structured box scene model is used for subsequent abnormal event propagation path construction, observability matrix generation, and terminal deployment optimization.

6. The multi-terminal self-booting deployment optimization method according to claim 1, characterized in that, In the structured box scene model, functional areas and sampling points where abnormal events may occur are defined, as well as feasible candidate terminal installation points under physical constraints. Construct an observability matrix that comprehensively characterizes the ability of any candidate terminal installation point to detect anomaly events at any location in space, including: In the structured box scene model, functional areas and sampling points where abnormal events may occur are defined, as well as candidate terminal installation points that are feasible under physical constraints. By calculating the direct propagation path between each event point and each installation point, and analyzing its intersection relationship with each bounding box, the equivalent occlusion degree and signal attenuation of the path are quantified, and an observability matrix that comprehensively describes the ability of any candidate point to perceive abnormal events at any location in space is generated. Based on the structured box scene model, the living space is divided into functional areas, and an abnormal event sampling point set is generated within the functional areas. The abnormal event sampling points are used to characterize the possible location distribution of abnormal events in the space. A set of candidate terminal deployment points is generated in the space, and the candidate terminal deployment points are filtered based on physical placement constraints; the physical placement constraints include at least placement plane constraints, preset height range constraints, and minimum safe distance constraints from noise sources. A spatial propagation path is constructed for each abnormal event sampling point and each candidate terminal deployment point; Based on propagation distance, occlusion counting model and reflection risk weight parameters, the perception capability of each abnormal event sampling point and candidate terminal deployment point is quantitatively evaluated, and an abnormal event observability matrix is ​​generated to characterize the perception distribution relationship of abnormal events to terminal deployment locations in space. A multi-terminal collaborative structure constraint model is constructed based on the observability matrix of the abnormal events. The collaborative structure constraint model is used to constrain the coverage complementarity relationship and redundancy suppression relationship among multiple terminals.

7. The multi-terminal self-booting deployment optimization method according to claim 1, characterized in that, The process of searching for the optimal terminal deployment combination among all candidate installation points based on the observability matrix and constrained by the number of terminals includes: Based on the observability matrix, with the number of terminals as a constraint, the optimal terminal deployment combination is searched among all candidate installation points. The search aims to maximize the overall observability of the system for all abnormal event sampling points, and takes the coverage complementarity and redundancy suppression between terminals as cooperative constraints. By solving this optimization problem, the optimal deployment scheme is output. Subsequently, by introducing perturbations into the model parameters, the robustness of the scheme under environmental parameter fluctuations is verified. Based on the set of candidate terminal deployment points and the constraint of the number of terminals, multiple candidate terminal deployment combinations are generated to construct a system-level deployment scheme space. Based on the observability matrix of abnormal events and the multi-terminal collaborative structure constraint model, the system-level perception capability of the candidate terminal deployment combinations is scored, and the candidate terminal deployment combinations are ranked according to the scoring results. The terminal deployment combination with the best score is selected as the optimal deployment scheme, and the abnormal event perception structure is reverse-analyzed based on the optimal deployment scheme to verify the consistency relationship between the deployment structure and the spatial distribution of abnormal events. The optimal deployment scheme is subjected to perturbation verification processing. By applying perturbations within a preset range to the occlusion weight parameter, reflection risk weight parameter and cooperative structure constraint parameter, the stability of the deployment scheme under environmental change conditions is evaluated.

8. A multi-terminal self-booting deployment optimization system, characterized in that, include: The data acquisition module is used to collect multi-view images and timestamp data of the living environment, and obtain the dataset after preprocessing; The model building module is used to build a structured box scene model representing the worst-case acoustic propagation conditions based on the dataset. The perception module is used to define functional areas and sampling points where abnormal events may occur in the structured box scene model, as well as feasible candidate terminal installation points under physical constraints, and to construct an observability matrix that comprehensively describes the ability of any candidate terminal installation point to perceive abnormal events at any location in space. The output module is used to search for the optimal combination of terminal deployments among all candidate installation points based on the observability matrix and with the number of terminals as a constraint.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the multi-terminal self-booting deployment optimization method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the multi-terminal self-booting deployment optimization method as described in any one of claims 1 to 7.

Images