Visualized IoT control method and system

Abstract

The embodiment of the present invention relates to the field of Internet of Things management and control technology, and specifically discloses a visual Internet of Things management and control method and system for managing equipment in a conference room, including: when a conference room debugging event is detected, loading a conference room model and displaying the conference room model on a display screen, wherein the target equipment used for the meeting is highlighted in the conference room model, obtaining position distribution feature data of participants in the conference room, obtaining environmental data of the conference room, and controlling the target equipment according to the position distribution feature data and the environmental data, thereby realizing a visual display of target equipment such as air conditioners, lights, and audio equipment in the conference room model, and being able to control the target equipment according to the position distribution feature data and environmental data of the participants, without the need for the participants to manually debug the equipment, and being able to flexibly control the equipment in the conference room according to the position distribution and environmental data of the participants, thereby improving the intelligence of the meeting.

Description

Technical Field

[0001] The present invention belongs to the technical field of Internet of Things control, and in particular relates to a visual Internet of Things control method and system. Background Art

[0002] In the smart conference system, conference room users can reserve conference rooms through the online conference room management system. For example, after entering the meeting time, number of participants and equipment to be used in the conference room management system, the system automatically reserves a suitable conference room for the conference user.

[0003] Currently, audio equipment, air conditioning, lighting and other equipment in conference rooms require participants to manually adjust them before the meeting, or set a unified adjustment plan in the system. The adjustment plan is single and cannot adjust related equipment according to the positions of participants in the meeting. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a visual IoT management and control method and system, which aims to solve the problem that participants need to manually adjust conference room equipment and cannot adjust related equipment according to the positions of participants in the meeting.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions: A visual IoT control method is used to control devices in a conference room, and specifically includes the following steps: When a conference room debugging event is detected, a conference room model is loaded and displayed on a display screen, wherein a target device used for the conference is highlighted in the conference room model; Obtaining location distribution feature data of participants in the conference room; Acquire environmental data of the conference room; The target device is controlled according to the location distribution feature data and the environment data, and the target device includes at least one of air conditioning, lighting, and audio equipment.

[0006] As a further limitation of the technical solution of the embodiment of the present invention, when a conference room debugging event is detected, before loading the conference room model and displaying the conference room model on the display screen, the following steps are specifically included: Obtaining reservation information for a conference room, wherein the reservation information includes at least the meeting time and the number of participants; Setting the debugging time according to the meeting time, or setting the number threshold according to the number of participants; The system time reaching the debugging time is regarded as a debugging event, or the number of people in the conference room reaches the number threshold is regarded as a debugging event; Determine whether the system time reaches the debugging time, or determine whether the number of people in the conference room is greater than or equal to the number threshold; If so, it is determined that a conference room debugging event is detected, and the steps of loading the conference room model and displaying the conference room model on a display screen are executed.

[0007] As a further limitation of the technical solution of the embodiment of the present invention, when a conference room debugging event is detected, loading a conference room model and displaying the conference room model on a display screen further includes the following steps: When a conference room debugging event is detected, determining the conference room ID of the conference room from the reservation information of the conference room; Loading a target model matching the conference room ID from a model library, and displaying the target model on a display screen as a conference room model of the conference room; Acquire device usage information from the predetermined information, and determine a target device based on the device usage information; The target device is highlighted in the displayed conference room model.

[0008] As a further limitation of the technical solution of the embodiment of the present invention, the conference room is provided with a color camera and an infrared camera, and obtaining the position distribution feature data of the participants in the conference room specifically includes the following steps: Controlling the color camera to collect color images of the conference room, and controlling the infrared camera to collect infrared images of the conference room; Inputting the color image and the infrared image into a preset participant recognition model to obtain the detection frame position of each participant; Converting the detection frame position into the actual position of each participant in the conference room; The location distribution characteristic data of the participants is calculated based on the actual location of each participant.

[0009] As a further limitation of the technical solution of the embodiment of the present invention, the participant recognition model includes a color image feature extraction subnetwork, an infrared image feature extraction subnetwork, a feature fusion subnetwork, a first detection subnetwork, a second detection subnetwork and an output subnetwork. The color image and the infrared image are input into a preset participant recognition model to obtain the detection frame position of each participant, which specifically includes the following steps: Extracting a first image feature from the color image in the color image feature extraction subnetwork; Extracting a second image feature from the infrared image in the infrared image feature extraction subnetwork; The first image feature and the second image feature are fused in the feature fusion subnetwork to obtain a fused feature; Recognize the fused features in the first detection sub-network to obtain a first detection frame position of the conference participant in the color image; The second detection sub-network recognizes the fused feature to obtain a second detection frame position of the conference participant in the infrared image; The first detection frame position and the second detection frame position are processed in the output subnetwork to obtain the detection frame positions of the conference participants.

[0010] As a further limitation of the technical solution of the embodiment of the present invention, calculating the location distribution characteristic data of the participants according to the real location of each participant specifically includes the following steps: For each first participant, the actual positions of the first participant and the second participant are used to determine the number of adjacent persons of the second participant within a preset range of the first participant, wherein the second participant is a person other than the first participant; Sort the first conference participants according to the number of adjacent participants from largest to smallest to obtain a candidate center conference participant sequence; Determine the participant ranked first in the candidate center participant sequence as the candidate center participant; Calculate the ratio of the number of adjacent participants of the candidate center to the total number of participants; Determining whether the ratio is greater than a preset threshold; If yes, the candidate center attendees are determined as target center attendees; If not, deleting the first ranked participant from the candidate center participant sequence, and returning to the step of determining the first ranked participant in the candidate center participant sequence as the candidate center participant; Generate a personnel distribution map according to the actual positions of each participant; The personnel distribution map and the actual positions of the target center participants are determined as the position distribution feature data of the participants.

[0011] As a further limitation of the technical solution of the embodiment of the present invention, the target device includes an audio device, and controlling the target device according to the location distribution feature data and the environment data specifically includes the following steps: Displaying the personnel distribution map in the conference room model and highlighting the locations of the target center participants; Displaying a reference sound field of the audio equipment and a sound field confirmation operation in the conference room model, wherein the sound field center of the reference sound field coincides with the actual position of the participant in the target center; Determining whether a sound field confirmation operation is detected; If yes, controlling the audio device so that the sound field center of the audio device coincides with the actual position of the participant at the target center; If not, when a sound field adjustment operation is received, the audio device is controlled according to the sound field adjustment operation.

[0012] As a further limitation of the technical solution of the embodiment of the present invention, the target device includes an air conditioner and a lighting lamp, the environmental data includes an ambient temperature and an ambient brightness, and the controlling the target device according to the position distribution characteristic data and the environmental data specifically includes the following steps: Displaying the air outlet of the air conditioner, lighting, personnel distribution map, and the ambient temperature and brightness of each participant's location in the personnel distribution map in the conference room model; When receiving the selection operation of the air-conditioning outlet, controlling the selected target air-conditioning outlet to open, and controlling the air volume of the outlet according to the ambient temperature; When a selection operation of a lighting lamp is received, the selected lighting lamp is controlled to be turned on, and the brightness of the lighting lamp is controlled according to the ambient brightness.

[0013] As a further limitation of the technical solution of the embodiment of the present invention, after controlling the target device according to the location distribution feature data and the environment data, the following steps are specifically included: Displaying the state of the target device in the conference room model, the state including on and off; Determine whether the status of all target devices is turned on; If yes, determine that the conference room is ready, and generate prompt information indicating that the conference room is ready; If not, determining that the conference room is not ready, generating an alarm message of an unactivated target device in the conference room model; When a conference end event is detected, determining whether the state of the target device is a closed state; If so, modify the state of the conference room to be idle; If not, the target device that is in the on state is controlled to be turned off.

[0014] A visual IoT management and control system is used to manage and control the equipment in the conference room, and specifically includes the following units: A conference room model loading and display unit, used to load the conference room model and display the conference room model on the display screen when a conference room debugging event is detected, wherein the conference room model highlights the target device used for the conference; A location distribution characteristic data acquisition unit, used to acquire location distribution characteristic data of conference participants in the conference room; An environmental data acquisition unit, used to acquire environmental data of the conference room; A device control unit is used to control the target device according to the location distribution feature data and the environmental data, and the target device includes at least one of air conditioning, lighting, and audio equipment.

[0015] Compared with the prior art, the present invention has the following beneficial effects: When a conference room debugging event is detected, the embodiment of the present invention loads a conference room model and visually displays the conference room model on a display screen and highlights the target devices used for the meeting. After obtaining the position distribution feature data and environmental data of the participants in the conference room, the air conditioner, lighting, audio equipment and other target devices in the conference room are controlled according to the position distribution feature data and environmental data, thereby realizing a visual display of the target devices in the conference room model and being able to control the target devices according to the position distribution feature data and environmental data of the participants, without the need for the participants to manually debug the devices, and being able to flexibly control the devices in the conference room according to the position distribution and environmental data of the participants, thereby improving the intelligence of the meeting. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0017] Figure 1 A flowchart of a visualized IoT management and control method provided by an embodiment of the present invention is shown.

[0018] Figure 2 A flow chart for obtaining location distribution feature data provided by an embodiment of the present invention is shown.

[0019] Figure 3 An application architecture diagram of a visual IoT management and control system provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0021] Figure 1 The flowchart of the visualized IoT control method provided by the embodiment of the present invention is shown. Specifically, the visualized IoT control method of the embodiment of the present invention comprises the following steps: Step S101, when a conference room debugging event is detected, a conference room model is loaded and displayed on a display screen, wherein the conference room model highlights a target device used for the conference.

[0022] The visual IoT management and control method of this embodiment is used to manage and control the equipment in the conference room. The equipment in the conference room may include air conditioners, lighting, audio equipment, conference interactive whiteboards, projectors and other equipment. The equipment in the conference room can be connected to the Internet of Things through an IoT gateway, and various sensors can be set in the conference room, such as color cameras, infrared cameras, temperature sensors, light sensors, etc.

[0023] In one application scenario, after a conference room user reserves a conference room, the visualized IoT management and control method of an embodiment of the present invention can be used to debug the equipment in the reserved conference room before the meeting starts, so that various devices in the conference room can be debugged to a better state before the meeting starts.

[0024] In another application scenario, during the process in the conference room, the visualized IoT control method of the embodiment of the present invention can be used to control the devices in the conference room in real time according to the changes in the participants in the conference room.

[0025] Specifically, this embodiment can be applied to a conference room management system. The server of the conference room management system can obtain the reservation information of the conference room, which includes at least the meeting time and the number of participants. The debugging time can be set according to the meeting time, such as two minutes or one minute before the meeting start time as the debugging time, or a number threshold can be set according to the number of participants. For example, if the number of participants is 10, the number threshold can be 3 people. Further, the system time reaching the debugging time is regarded as a debugging event, or the number of people in the conference room reaches the number threshold as a debugging event, and it is judged whether the system time reaches the debugging time, or it is judged whether the number of people in the conference room is greater than or equal to the number threshold. If so, it is determined that a conference room debugging event is detected, and the conference room model is loaded and displayed on the display screen. The conference room debugging is triggered by the debugging time and the number threshold, thereby realizing automatic triggering of the conference room debugging, without the need for manual triggering of the debugging, and avoiding the conference room manager from missing the debugging of the conference room equipment.

[0026] Among them, the conference room model can be a plane model, such as a top view of the conference room. Preferably, the conference room model can be a 3D model. The conference room model of each conference room can be pre-configured in the conference room management system. When a conference room debugging event is detected, the conference room ID of the conference room can be determined from the reservation information of the conference room, and the target model matching the conference room ID can be loaded from the model library, and the target model can be displayed on the display screen as the conference room model of the conference room. The device usage information is obtained from the reservation information, and the target device is determined based on the device usage information. The target device is highlighted in the displayed conference room model. The highlighting can be a highlight display, such as displaying the target device in different colors (such as red or yellow).

[0027] The reservation information may include information such as the ID of the conference room reserved by the conference room user, the number of participants, the meeting theme, the meeting start time, the meeting end time, and the equipment used for the meeting. After the ID of the reserved conference room is determined from the reservation information, the corresponding conference room model can be loaded and displayed on the display screen. The target equipment required for the meeting can be further highlighted in the displayed conference room model. For example, the target device can be highlighted in red or yellow in the conference room model, and the name of the target device can be displayed, so that the conference room manager can determine each target device from the displayed conference room model.

[0028] Step S102, obtaining location distribution feature data of conference participants in the conference room.

[0029] The location distribution feature data may be data indicating the gathering center and location features of the participants in the conference room. In a scenario, in a large conference room, without limiting the seats of the participants, it is more likely that the participants will randomly find a seat after entering the conference room, resulting in an uncertain location distribution of the participants. In order to make the audio equipment, air conditioner, lighting and other equipment in the conference room more effective or more energy-efficient, it is necessary to control the equipment in the conference room through the location distribution feature data. Specifically, step S102 may include the following steps: Step S1021, controlling the color camera to collect color images of the conference room, and controlling the infrared camera to collect infrared images of the conference room.

[0030] The conference room of this embodiment may be provided with a color camera and an infrared camera. The color camera may be used to capture color images of the area of ​​the conference room participants, and the infrared camera may be used to capture infrared images of the area of ​​the conference room participants. By way of example, after the color camera and the infrared camera are time synchronized, the color camera and the infrared camera may be controlled to capture images of the area of ​​the conference room participants to obtain color images and infrared images.

[0031] Step S1022: input the color image and the infrared image into a preset participant recognition model to obtain the detection frame position of each participant.

[0032] The participant identification model can be a model that identifies the detection box position of the participant from the color image and the infrared image. The participant identification model can be pre-trained. For example, the participant identification model includes a color image feature extraction subnetwork, an infrared image feature extraction subnetwork, a feature fusion subnetwork, a first detection subnetwork, a second detection subnetwork and an output subnetwork, wherein the outputs of the color image feature extraction subnetwork and the infrared image feature extraction subnetwork are used as inputs to the feature fusion subnetwork, the outputs of the feature fusion subnetwork are used as inputs to the first detection subnetwork and the second detection subnetwork, and the outputs of the first detection subnetwork and the second detection subnetwork are used as inputs to the output subnetwork, so as to output the detection box position of the participant from the output subnetwork.

[0033] During training, the detection frames and positions of the participants can be marked in the color images and infrared images used for training. For details, reference can be made to the detection frame marking of target detection in the prior art; and the correspondence between the detection frames belonging to the same participant in the color images and infrared images is marked (for example, the detection frames of the same participant are marked with the same value). After the color images and infrared images used for training are input into the participant recognition model, the output subnetwork outputs the detection frame positions of the participants in the infrared images and the correspondence with the same participant. The loss rate is calculated by the output detection frame positions and correspondences and the marked detection frame positions and correspondences (the loss rate is calculated by the mean square error loss function and the cross entropy loss function), and then the model parameters are adjusted, and training is continued until the loss rate is less than a threshold or the number of iterative training reaches a preset number, and a trained participant recognition model can be obtained.

[0034] In practical applications, the collected color image and infrared image can be input into a preset participant recognition model to obtain the detection frame position of each participant. Specifically, after the color image and infrared image are input into the participant recognition model, the color image feature extraction subnetwork extracts the first image feature of the color image, and the infrared image feature extraction subnetwork extracts the second image feature of the infrared image. The first image feature and the second image feature are fused by the feature fusion subnetwork to obtain a fusion feature. The fusion feature is identified in the first detection subnetwork to obtain the first detection frame position of the participant in the color image. The fusion feature is identified in the second detection subnetwork to obtain the second detection frame position of the participant in the infrared image. The first detection frame position and the second detection frame position are processed in the output subnetwork to obtain the detection frame position of the participant. The output subnetwork is specifically used to determine the detection frame of the same participant from the first detection frame position and the second detection frame position, and retain the repeated detection frame in the infrared image. The obtained detection frame position can be the image position of the detection frame in the infrared image. Of course, the detection frame position can also be the image position of the detection frame in the color image.

[0035] This embodiment identifies the positions of conference participants through color images and infrared images. On the one hand, it can collect images from different angles to identify conference participants, thereby improving the accuracy of conference participant identification. On the other hand, identifying conference participants through infrared images can make up for the problem of inaccurate conference participant identification through color images in insufficient light, thereby improving the reliability of conference participant identification.

[0036] Step S1023: convert the position of the detection frame into the actual position of each participant in the conference room.

[0037] In one embodiment, the detection box position can be the position of the detection box in the image, for example, the position in the infrared image. The image position can be converted into the position of the infrared camera coordinate system, and then the position in the infrared camera coordinate system can be converted into the position in the conference room coordinate system, so as to obtain the real position of each participant, which can be the x, y coordinate data of the participant in the conference room.

[0038] Step S1024, calculating the location distribution characteristic data of the participants according to the actual location of each participant.

[0039] In one embodiment, for each first participant, the actual positions of the first participant and the second participant are used to determine the number of adjacent persons of the second participant within a preset range of the first participant, wherein the second participant is a person other than the first participant, and the first participants are sorted from large to small according to the number of adjacent persons to obtain a candidate center participant sequence, and the participant ranked first in the candidate center participant sequence is determined as the candidate center participant, and the ratio of the number of adjacent persons of the candidate center participant to the total number of participants is calculated to determine whether the ratio is greater than a preset threshold value. If so, the candidate center participant is determined as the target center participant, if not, the participant ranked first is deleted from the candidate center participant sequence, and the step of determining the participant ranked first in the candidate center participant sequence as the candidate center participant is returned to generate a personnel distribution map according to the actual positions of each participant, and the personnel distribution map and the actual positions of the target center participants are determined as the position distribution feature data of the participants.

[0040] For example, assuming that there are 20 participants A1-A20, for participant A1, the distances between participant A1 and participants A2-A20 are calculated respectively through the real position (x, y coordinates), and then the number of participants whose distances to participant A1 are within the preset range is determined to obtain the number of adjacent persons. For example, the number of participants within the preset range of participant A1 is 9, and so on, the number of participants within the preset range of participant A2 is 8, and the number of participants within the preset range of participant A3 is 15. The number of adjacent persons within the preset range of each participant is obtained, and then the corresponding The participants are sorted by the number of neighbors to obtain a candidate center participant sequence. For example, the candidate center participant sequence is (A3, A1, A2, A10...A8), and then the ratio of the number of neighbors of participant A3 to the total number of participants is calculated. If the ratio is greater than a preset threshold, participant A3 is determined as the target center participant. If not, participant A3 is deleted from the candidate center participant sequence, and participant A1 is the first participant in the ranking. The ratio of the number of neighbors of participant A3 to the total number of participants is calculated, and so on, until the target center participant is determined. If the target center participant is not determined, the participant with the largest ratio of the number of neighbors to the total number of participants is determined as the target center participant, or the participant closest to the geometric center of the conference room is determined as the target center participant.

[0041] This embodiment determines the gathering center of the participants by the ratio of the number of adjacent participants within a preset range to the total number of participants. The calculation method is simple and the amount of calculation is small. It can accurately and quickly determine the gathering center of the participants, thereby improving the management and control efficiency of the conference room equipment. Of course, those skilled in the art can also cluster the real positions (x, y coordinates) of the participants through a clustering algorithm, and determine the participants corresponding to the cluster center, or the participants with the smallest distance from the cluster center as the target center participants.

[0042] The personnel distribution map can be a schematic diagram of the actual positions of the personnel on the x, y plane (the floor of the conference room) of the conference room model, and the personnel distribution map and the actual positions of the target center participants are determined as the position distribution feature data of the participants.

[0043] Step S103, obtaining the environment data of the conference room.

[0044] In this embodiment, multiple sensors connected to the Internet of Things gateway can be set up in the conference room. The sensors may include a temperature sensor for obtaining the ambient temperature and a light sensor for obtaining the ambient light brightness. The ambient temperature of multiple areas in the conference room can be collected through the temperature sensor, and the ambient brightness of multiple areas in the conference room can be obtained through the light sensor.

[0045] Step S104, controlling the target device according to the location distribution feature data and the environment data.

[0046] In one embodiment, the target device may include an audio device, which can display a personnel distribution map and highlight the location of the target center participant in the displayed conference room model, as well as display a reference sound field and a sound field confirmation operation of the audio device in the conference room model. The sound field center of the reference sound field coincides with the actual position of the target center participant, and a determination is made as to whether a sound field confirmation operation is detected. If so, the audio device is controlled so that the sound field center of the audio device coincides with the actual position of the target center participant. If not, when a sound field adjustment operation is received, the audio device is controlled according to the sound field adjustment operation.

[0047] Specifically, the audio equipment may include multiple speakers that can adjust the position and angle. After superimposing the personnel distribution map and the position of the target center participants on the conference room model displayed on the display screen, a reference sound field map can be superimposed, and the sound field center of the reference sound field map coincides with the position of the target center participants, and a sound field confirmation button is displayed. If the conference room manager clicks the confirmation button, it is deemed that the conference room manager accepts to control the audio equipment according to the reference sound field map so that the sound field center of the audio equipment coincides with the actual position of the target center participants, such as controlling multiple speakers to adjust the position and angle. This embodiment does not limit the algorithm for controlling multiple speakers to adjust the position and angle so that the sound field center of the audio equipment coincides with the actual position of the target center participants. Reference can be made to the prior art and will not be described in detail here. If the conference room manager does not click the confirmation button within the preset time length, or clicks the manual adjustment button, the sound field center graphic element is displayed on the display interface, so that the conference room manager can drag the sound field center graphic element in the conference room model to adjust the sound field center of the audio equipment.

[0048] In another embodiment, the target device may include air conditioners and lighting lamps. The air-conditioning outlets, lighting lamps, a personnel distribution map, and the ambient temperature and ambient brightness of each participant's location in the personnel distribution map may be displayed in the displayed conference room model. When a selection operation for the air-conditioning outlet is received, the selected target air-conditioning outlet is controlled to be turned on, and the air volume of the outlet is controlled according to the ambient temperature. When a selection operation for the lighting lamp is received, the selected lighting lamp is controlled to be turned on, and the brightness of the lighting lamp is controlled according to the ambient brightness. Specifically, the displayed air-conditioning outlets and lighting lamps may be regarded as a control. Taking the air-conditioning outlet as an example, the conference room manager may select the corresponding air-conditioning outlet from the interactive interface according to the location of the participant in the personnel distribution map, such as selecting the outlet close to the participant to open the air-conditioning outlet. The air-conditioning system adjusts the air volume of the air-conditioning outlet according to the ambient temperature of the participant's location and the preset target temperature. Similarly, the conference room manager may select the corresponding lighting lamp to turn on, so that the lighting system adjusts the brightness of the lighting lamp according to the ambient brightness of the locations of different participants.

[0049] This embodiment displays a personnel distribution map, target center participants, and target devices on a display screen, thereby presenting a visual operation interface to conference room managers, so that the conference room managers can control the target devices in the conference room according to the distribution of participants through the visual interface.

[0050] Of course, the system can also select corresponding lights and air-conditioning vents to turn on. For example, it can select lights and air-conditioning vents closest to the participants to turn on, without the need for manual operation, thus improving the degree of automation.

[0051] In another embodiment, the status of the target device can also be displayed in the conference room model, the status includes on and off, and it is determined whether the status of all target devices is on. If so, it is determined that the conference room is ready, and a prompt message that the conference room preparation is complete is generated. If not, it is determined that the conference room is not ready, and an alarm message of the unactivated target device is generated in the conference room model, thereby realizing automatic detection of whether the target devices required for the meeting are fully ready, avoiding under-adjustment or misadjustment of conference room equipment.

[0052] In one embodiment, when a meeting end event is detected, it can also be determined whether the state of the target device is closed. If so, the state of the conference room is changed to an idle state. If not, the target device in the open state is controlled to be closed. The meeting end event is a preset time after the meeting end time, or a preset time after no one is detected in the conference room. By monitoring the state of the target device, all target devices can be closed after the meeting, avoiding the problem of energy waste caused by participants not closing the target devices after the meeting.

[0053] When a conference room debugging event is detected, the embodiment of the present invention loads a conference room model and visually displays the conference room model on a display screen and highlights the target devices used for the meeting. After obtaining the position distribution feature data and environmental data of the participants in the conference room, the air conditioner, lighting, audio equipment and other target devices in the conference room are controlled according to the position distribution feature data and environmental data, thereby realizing a visual display of the target devices in the conference room model and being able to control the target devices according to the position distribution feature data and environmental data of the participants, without the need for the participants to manually debug the devices, and being able to flexibly control the devices in the conference room according to the position distribution and environmental data of the participants, thereby improving the intelligence of the meeting.

[0054] Figure 3 The application architecture diagram of the visual IoT management and control system provided by an embodiment of the present invention is shown. The visual IoT management and control system of this embodiment specifically includes the following units: The conference room model loading and display unit 301 is used to load the conference room model and display the conference room model on the display screen when a conference room debugging event is detected, wherein the conference room model highlights the target device used for the conference; A location distribution characteristic data acquisition unit 302 is used to acquire location distribution characteristic data of conference participants in the conference room; An environmental data acquisition unit 303, used to acquire environmental data of the conference room; The device control unit 304 is used to control the target device according to the location distribution feature data and the environmental data, and the target device includes at least one of air conditioning, lighting, and audio equipment.

[0055] As a further limitation of the technical solution of the embodiment of the present invention, before the conference room model is loaded into the display unit 301, the following units are specifically included: A reservation information acquisition unit, used to acquire reservation information of the conference room, wherein the reservation information at least includes the meeting time and the number of participants; A condition setting unit, used to set the debugging time according to the meeting time, or to set the number threshold according to the number of participants; A debugging event setting unit, used to set the system time reaching the debugging time as a debugging event, or the number of people in the conference room reaching the number threshold as a debugging event; A debugging event judgment unit, used to judge whether the system time reaches the debugging time, or to judge whether the number of people in the conference room is greater than or equal to the number threshold, and if so, to execute the determination unit; The determination unit is used to determine that a conference room debugging event is detected and execute the conference room model loading and display unit 301.

[0056] As a further limitation of the technical solution of the embodiment of the present invention, the conference room model loading and display unit 301 specifically includes the following modules: A conference room ID determination module, used to determine the conference room ID of the conference room from the reservation information of the conference room when a conference room debugging event is detected; A conference room model display module, used to load a target model matching the conference room ID from a model library and display the target model on a display screen as a conference room model of the conference room; a target device determination module, configured to obtain device usage information from the predetermined information and determine a target device based on the device usage information; The target device highlighting module is used to highlight the target device in the displayed conference room model.

[0057] As a further limitation of the technical solution of the embodiment of the present invention, the location distribution feature data acquisition unit 302 specifically includes the following modules: An image acquisition module, used to control the color camera to acquire a color image of the conference room, and to control the infrared camera to acquire an infrared image of the conference room; A detection frame position determination module, used for inputting the color image and the infrared image into a preset participant recognition model to obtain the detection frame position of each participant; A real position conversion module, used to convert the detection frame position into the real position of each participant in the conference room; The location feature distribution data calculation module is used to calculate the location distribution feature data of the participants according to the actual location of each participant.

[0058] As a further limitation of the technical solution of the embodiment of the present invention, the participant recognition model includes a color image feature extraction subnetwork, an infrared image feature extraction subnetwork, a feature fusion subnetwork, a first detection subnetwork, a second detection subnetwork and an output subnetwork, and the position feature distribution data calculation module specifically includes the following submodules: A first image feature extraction submodule, used for extracting a first image feature from the color image in the color image feature extraction subnetwork; A second image feature extraction submodule, used for extracting a second image feature from the infrared image in the infrared image feature extraction subnetwork; A feature fusion submodule, used for fusing the first image feature and the second image feature in the feature fusion subnetwork to obtain a fused feature; A first detection frame position recognition submodule, configured to recognize the fused feature in the first detection subnetwork to obtain a first detection frame position of the conference participant in the color image; A second detection frame position recognition submodule, configured to recognize the fused feature in the second detection subnetwork to obtain a second detection frame position of the conference participant in the infrared image; The detection frame position processing submodule is used to process the first detection frame position and the second detection frame position in the output subnetwork to obtain the detection frame position of the participant.

[0059] As a further limitation of the technical solution of the embodiment of the present invention, the location feature distribution data calculation module specifically includes the following submodules: a neighboring person number calculation submodule, for determining, for each first participant, the number of neighbors of the second participant within a preset range of the first participant using the actual positions of the first participant and the second participant, wherein the second participant is a person other than the first participant; A participant sequence generation submodule, used to sort the first participants from largest to smallest according to the number of adjacent participants, to obtain a candidate center participant sequence; A candidate center participant determination submodule is used to determine the participant ranked first in the candidate center participant sequence as the candidate center participant; A ratio calculation submodule, used to calculate the ratio of the number of adjacent participants of the candidate center participant to the total number of participants; The ratio judgment submodule is used to judge whether the ratio is greater than a preset threshold value. If so, the target center participant determination submodule is executed; if not, the candidate center participant update submodule is executed; A target center participant determination submodule is used to determine the candidate center participant as the target center participant; A candidate center participant updating submodule, used to delete the first ranked participant from the candidate center participant sequence and return to the candidate center participant determination submodule; A personnel distribution map generating submodule is used to generate a personnel distribution map according to the actual positions of each participant; The position distribution characteristic data determination submodule is used to determine the personnel distribution map and the actual positions of the target center participants as the position distribution characteristic data of the participants.

[0060] As a further limitation of the technical solution of the embodiment of the present invention, the target device includes an audio device, and the device control unit 304 specifically includes the following modules: A participant data display module, used to display the personnel distribution map in the conference room model and highlight the location of the central participant; A reference sound field display module, used to display the reference sound field of the audio equipment and the sound field confirmation operation in the conference room model, wherein the sound field center of the reference sound field coincides with the real position of the central conference participant; A sound field operation determination module, used to determine whether a sound field confirmation operation is detected, and if so, execute the first audio device control module, and if not, execute the second audio device control module; A first audio device control module, used to control the audio device so that the sound field center of the audio device coincides with the actual position of the central conference participant; The second audio device control module is used to control the audio device according to the sound field adjustment operation when receiving the sound field adjustment operation.

[0061] As a further limitation of the technical solution of the embodiment of the present invention, the target device includes an air conditioner and a lighting lamp, the environmental data includes an ambient temperature and an ambient brightness, and the device control unit 304 specifically includes the following modules: A display module is used to display the air outlet of the air conditioner, the lighting, the personnel distribution map, and the ambient temperature and brightness of the location of each participant in the personnel distribution map in the conference room model; An air outlet control module, for controlling the selected target air outlet to open when receiving an air-conditioning air outlet selection operation, and controlling the air volume of the air outlet according to the ambient temperature; The lighting control module is used to control the selected lighting to turn on when receiving the lighting selection operation, and to control the brightness of the lighting according to the ambient brightness.

[0062] As a further limitation of the technical solution of the embodiment of the present invention, the following units are specifically included: A device status display unit, used to display the status of the target device in the conference room model, wherein the status includes on and off; A first device status determination unit, used to determine whether the status of all target devices is turned on, if so, execute the first determination unit, if not, execute the second determination unit; A first determining unit, configured to determine that the conference room is ready, and generate prompt information indicating that the conference room is ready; A second determining unit, configured to determine that the conference room is not ready, and generate warning information of an unactivated target device in the conference room model; A second device status judgment unit is used to judge whether the state of the target device is a closed state when a conference end event is detected, and if so, execute the conference room status modification unit, and if not, execute the device closing unit; A conference room status modification unit, used to modify the status of the conference room to an idle state; The device closing unit is used to control the target device in the turned-on state to close.

[0063] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0064] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0065] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0066] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the attached claims.

[0067] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A visual IoT control method, characterized in that: Used to control the equipment in the conference room, including the following steps: When a conference room debugging event is detected, a conference room model is loaded and displayed on a display screen, wherein a target device used for the conference is highlighted in the conference room model; Obtaining location distribution feature data of participants in the conference room; Acquire environmental data of the conference room; The target device is controlled according to the location distribution feature data and the environment data, and the target device includes at least one of air conditioning, lighting, and audio equipment.

2. The visual IoT control method according to claim 1, characterized in that: When a conference room debugging event is detected, the conference room model is loaded and before the conference room model is displayed on the display screen, the following steps are specifically included: Obtaining reservation information for a conference room, wherein the reservation information includes at least the meeting time and the number of participants; Setting the debugging time according to the meeting time, or setting the number threshold according to the number of participants; The system time reaching the debugging time is regarded as a debugging event, or the number of people in the conference room reaches the number threshold is regarded as a debugging event; Determine whether the system time reaches the debugging time, or determine whether the number of people in the conference room is greater than or equal to the number threshold; If so, it is determined that a conference room debugging event is detected, and the steps of loading a conference room model and displaying the conference room model on a display screen are executed.

3. The visual IoT control method according to claim 2, characterized in that: When a conference room debugging event is detected, loading a conference room model and displaying the conference room model on a display screen specifically includes the following steps: When a conference room debugging event is detected, determining the conference room ID of the conference room from the reservation information of the conference room; Loading a target model matching the conference room ID from a model library, and displaying the target model on a display screen as a conference room model of the conference room; Acquire device usage information from the predetermined information, and determine a target device based on the device usage information; The target device is highlighted in the displayed conference room model.

4. The visual IoT control method according to claim 2, characterized in that: The conference room is provided with a color camera and an infrared camera, and obtaining the position distribution characteristic data of the participants in the conference room specifically includes the following steps: Controlling the color camera to collect color images of the conference room, and controlling the infrared camera to collect infrared images of the conference room; Inputting the color image and the infrared image into a preset participant recognition model to obtain the detection frame position of each participant; Converting the detection frame position into the actual position of each participant in the conference room; The location distribution characteristic data of the participants is calculated based on the actual location of each participant.

5. The visual IoT control method according to claim 4, characterized in that: The participant recognition model includes a color image feature extraction subnetwork, an infrared image feature extraction subnetwork, a feature fusion subnetwork, a first detection subnetwork, a second detection subnetwork, and an output subnetwork. The color image and the infrared image are input into the preset participant recognition model to obtain the detection frame position of each participant, which specifically includes the following steps: Extracting a first image feature from the color image in the color image feature extraction subnetwork; Extracting a second image feature from the infrared image in the infrared image feature extraction subnetwork; The first image feature and the second image feature are fused in the feature fusion subnetwork to obtain a fused feature; Recognize the fused features in the first detection sub-network to obtain a first detection frame position of the conference participant in the color image; The second detection sub-network recognizes the fused feature to obtain a second detection frame position of the conference participant in the infrared image; The first detection frame position and the second detection frame position are processed in the output subnetwork to obtain the detection frame positions of the conference participants.

6. The visual IoT control method according to claim 4, characterized in that: Calculating the location distribution feature data of the participants according to the actual location of each participant includes the following steps: For each first participant, the actual positions of the first participant and the second participant are used to determine the number of adjacent persons of the second participant within a preset range of the first participant, wherein the second participant is a person other than the first participant; Sort the first conference participants according to the number of adjacent participants from largest to smallest to obtain a candidate center conference participant sequence; Determine the participant ranked first in the candidate center participant sequence as the candidate center participant; Calculate the ratio of the number of adjacent participants of the candidate center to the total number of participants; Determining whether the ratio is greater than a preset threshold; If yes, the candidate center attendees are determined as target center attendees; If not, deleting the first ranked participant from the candidate center participant sequence, and returning to the step of determining the first ranked participant in the candidate center participant sequence as the candidate center participant; Generate a personnel distribution map according to the actual positions of each participant; The personnel distribution map and the actual positions of the target center participants are determined as the position distribution feature data of the participants.

7. The visual IoT control method according to claim 6, characterized in that: The target device includes an audio device, and controlling the target device according to the location distribution feature data and the environment data specifically includes the following steps: Displaying the personnel distribution map in the conference room model and highlighting the locations of the target center participants; Displaying a reference sound field of the audio equipment and a sound field confirmation operation in the conference room model, wherein the sound field center of the reference sound field coincides with the actual position of the participant in the target center; Determining whether a sound field confirmation operation is detected; If yes, controlling the audio device so that the sound field center of the audio device coincides with the actual position of the participant at the target center; If not, when a sound field adjustment operation is received, the audio device is controlled according to the sound field adjustment operation.

8. The visual IoT control method according to claim 6, characterized in that: The target device includes an air conditioner and a lighting lamp, the environmental data includes an environmental temperature and an environmental brightness, and the control of the target device according to the position distribution characteristic data and the environmental data specifically includes the following steps: Displaying the air outlet of the air conditioner, lighting, personnel distribution map, and the ambient temperature and brightness of the location of each participant in the personnel distribution map in the conference room model; When receiving the selection operation of the air-conditioning outlet, controlling the selected target air-conditioning outlet to open, and controlling the air volume of the outlet according to the ambient temperature; When a selection operation of a lighting lamp is received, the selected lighting lamp is controlled to be turned on, and the brightness of the lighting lamp is controlled according to the ambient brightness.

9. The visual IoT control method according to any one of claims 1 to 8, characterized in that: After controlling the target device according to the location distribution characteristic data and the environment data, the method further includes the following steps: Displaying the state of the target device in the conference room model, the state including on and off; Determine whether the status of all target devices is turned on; If yes, determine that the conference room is ready, and generate prompt information indicating that the conference room is ready; If not, determining that the conference room is not ready, generating an alarm message of an unactivated target device in the conference room model; When a conference end event is detected, determining whether the state of the target device is a closed state; If so, modify the state of the conference room to be idle; If not, the target device that is in the on state is controlled to be turned off.

10. A visual IoT management and control system, characterized in that: Used to control the equipment in the conference room, including the following units: A conference room model loading and display unit, used to load the conference room model and display the conference room model on the display screen when a conference room debugging event is detected, wherein the conference room model highlights the target device used for the conference; A location distribution characteristic data acquisition unit, used to acquire location distribution characteristic data of conference participants in the conference room; An environmental data acquisition unit, used to acquire environmental data of the conference room; A device control unit is used to control the target device according to the location distribution feature data and the environmental data, and the target device includes at least one of air conditioning, lighting, and audio equipment.

Images