Linkage control method and system

Abstract

The invention provides a linkage control method and system, and belongs to the field of display screen linkage control, and the method comprises the steps: obtaining an echo signal which is reflected in a target scene and is received by a microwave radar sensor in real time, and detecting whether there is a person in the target scene or not according to the echo signal; collecting sound in the target scene through a sound sensor, and judging whether the collected sound is effective sound or not; if the sound in the target scene collected by the sound sensor is effective sound, determining a movement trend range of people in the target scene according to the image sensor and the microwave radar sensor; the static state of the personnel in the target scene is judged according to the movement trend range of the personnel in the target scene, a screen projection starting instruction is sent to the display screen controller through the microwave radar sensor, and the display screen is controlled to be started. According to the linkage control method and system provided by the invention, the movement trend range can be determined according to the movement state of the personnel in the target scene, and linkage control is performed on the display screen according to the movement trend range.

Description

Technical Field

[0001] The present application belongs to the technical field of display screen linkage control, and more specifically, relates to a linkage control method and system. Background Art

[0002] In the field of display linkage control, the primary reliance on single sensors or fixed time rules for display control presents numerous limitations. Some single-sensor control methods, such as those that rely solely on infrared sensors to detect human presence to control display on / off, can easily misjudge the presence of stationary users as absent, causing the display to unnecessarily shut down. Single sensors also have a poor ability to distinguish between human activity and environmental interference, often misinterpreting environmental disturbances such as swaying objects as human activity, leading to the inadvertent issuance of display control commands.

[0003] Therefore, when this method of identifying the movement status of people in the target scene based on a single sensor, the recognition accuracy is low, and the display screen cannot be flexibly controlled according to the movement status of people. It is difficult to accurately adapt to the linkage requirements between the status of people and the display screen in complex scenes. Summary of the Invention

[0004] The purpose of this application is to provide a linkage control method and system, which uses microwave radar sensors, sound sensors, and image sensors to collect the motion status of people in the target scene, determine the motion trend range based on the motion status, and send corresponding control instructions to the display screen controller based on the motion trend range to achieve linkage control of the display screen.

[0005] In a first aspect of an embodiment of the present application, a linkage control method is provided, which is applied to a linkage control device, wherein the linkage control device includes a microwave radar sensor, a sound sensor, an image sensor, and a display controller. The method includes: Acquire an echo signal reflected from a target scene received in real time by a microwave radar sensor, and determine that a person is present in the target scene if the amplitude change of the echo signal acquired two times consecutively within a first acquisition time interval is greater than or equal to a preset amplitude change threshold; If the amplitude variation of the echo signal collected within the first collection time interval is less than a preset amplitude variation threshold, it is determined that no one is in the target scene, and the preset amplitude variation threshold is a threshold for distinguishing between human body and environmental interference; The sound sensor is used to collect the sound in the target scene and determine whether the collected sound is a valid sound; If the sound collected by the sound sensor in the target scene is a valid sound, the movement trend range of the person in the target scene is determined according to the image sensor and the microwave radar sensor; If the motion trend range is less than or equal to the first preset range, it is determined that the person in the target scene is in a stationary state, and a projection screen start instruction is sent to the display screen controller through the microwave radar sensor to control the display screen to turn on; if the motion trend range is greater than the first preset range, it is determined that the person in the target scene is in an active state, and the microwave radar sensor sends a projection screen standby instruction to the display screen controller.

[0006] A second aspect of the embodiments of the present application provides a linkage control system, including: A microwave signal acquisition module is configured to acquire, in real time, an echo signal reflected from a target scene received by a microwave radar sensor. If the amplitude change of two consecutive echo signals acquired within a first acquisition time interval is greater than or equal to a preset amplitude change threshold, it is determined that a person is present in the target scene. If the amplitude change of the echo signal acquired within the first acquisition time interval is less than the preset amplitude change threshold, it is determined that no one is present in the target scene. The preset amplitude change threshold is a threshold for distinguishing between human interference and environmental interference. A sound judgment module is used to collect the sound in the target scene through the sound sensor and determine whether the collected sound is valid sound; The command control module is used to collect the sound in the target scene as valid sound based on the sound sensor, and determine the movement trend range of people in the target scene based on the image sensor and microwave radar sensor; If the motion trend range is less than or equal to the first preset range, it is determined that the person in the target scene is in a stationary state, and a projection screen start instruction is sent to the display screen controller through the microwave radar sensor to control the display screen to turn on; if the motion trend range is greater than the first preset range, it is determined that the person in the target scene is in an active state, and the microwave radar sensor sends a projection screen standby instruction to the display screen controller.

[0007] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the above-mentioned linkage control method when executing the computer program.

[0008] In a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned linkage control method are implemented.

[0009] The beneficial effects of a linkage control method and system provided by an embodiment of the present application are as follows: the embodiment of the present application receives the echo signal reflected in the target scene in real time through a microwave radar sensor, dynamically compares the echo difference in adjacent time intervals by collecting the amplitude change of the echo signal collected twice in a time interval, ensures the continuity and stability of the echo signal, accurately distinguishes the interference between the human body and the environment, and reduces the misjudgment caused by instantaneous interference; by setting a preset sound decibel value, effectively distinguishes the environmental background noise from the target effective sound, reduces the misjudgment caused by low-intensity noise, and improves the stability of the effective sound judgment; by combining the data of the three sensors of sound, image and microwave radar, realizes the collaborative work of multimodal sensors and improves the judgment accuracy; determines the movement trend range of the people in the target scene according to the movement state of the people in the target scene, sends corresponding control instructions to the display screen controller according to the movement trend range, and realizes linkage control of the display screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0011] Figure 1 A flowchart of a linkage control method provided in one embodiment of the present application; Figure 2 A structural block diagram of a linkage control system provided in one embodiment of the present application; Figure 3 A schematic block diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0012] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0013] In order to make the purpose, technical solutions and advantages of this application clearer, specific embodiments will be described below with reference to the accompanying drawings.

[0014] Please refer to Figure 1 , Figure 1A flowchart of a linkage control method provided in one embodiment of the present application may be performed by a linkage control device, which may include a microwave radar sensor, a sound sensor, an image sensor, and a display controller. The method may include: S101: Acquire an echo signal reflected from a target scene received in real time by a microwave radar sensor. If a change in the amplitude of two consecutive echo signals acquired within a first acquisition time interval is greater than or equal to a preset amplitude change threshold, determine that a person is in the target scene. If the amplitude variation of the echo signal collected within the first collection time interval is less than a preset amplitude variation threshold, it is determined that there is no person in the target scene. The preset amplitude variation threshold is a threshold for distinguishing between human body and environmental interference.

[0015] In this embodiment, the first acquisition time interval is set to 2 seconds, and the preset amplitude change threshold is 0.3V. The preset amplitude change threshold is the threshold for distinguishing between human body and environmental interference. According to historical data, the amplitude change of the human body reflected echo is ≥0.3V, while the amplitude change of the reflected echo caused by environmental interference such as fluttering curtains and slight shaking of the equipment is ≤0.2V.

[0016] The microwave radar sensor continuously transmits electromagnetic waves and receives reflected echo signals. The amplitude change of two consecutive echo signals collected within a first acquisition time interval is determined. The presence of a person in the target scene is determined based on the amplitude change and a preset amplitude change threshold. If the amplitude change of the echo signals collected twice within the first acquisition time interval is greater than or equal to the preset amplitude change threshold, the target scene is determined to be occupied. If the amplitude change of the echo signals collected within the first acquisition time interval is less than the preset amplitude change threshold, the target scene is determined to be unoccupied. The target scene may be a conference room or exhibition hall, for example. This embodiment further provides a second acquisition time interval of 5 seconds. After determining whether a person is present in the target scene, at each second acquisition time interval, the microwave radar sensor again determines whether a person is present in the target scene based on two consecutive echo signals collected within the first acquisition time interval. If the amplitude change of the echo signals is less than the preset amplitude change threshold for a predetermined number of consecutive times, the conference room is determined to be unoccupied. In this embodiment, the predetermined number of times is 3.

[0017] For example, if the microwave radar sensor receives an echo signal of 1.2V for the first time and 1.6V for the second time within a 2-second interval, the amplitude change between the two consecutive echo signals is 0.4V, which is greater than the preset amplitude change threshold. This indicates the presence of a person in the conference room. After determining the presence of a person in the conference room, a second acquisition interval of 5 seconds is set. Every 5 seconds, the microwave radar sensor uses the echo signals collected twice within the first acquisition interval to determine whether a person is present. If the amplitude change of the echo signal is less than the preset amplitude change threshold three times in a row, the conference room is determined to be empty. This example can filter out environmental interference, preventing the display screen from activating inadvertently due to interference. It can also adapt to complex human states, accurately identifying both moving and stationary individuals, addressing the drawback of a single sensor that "recognizes movement but misidentifies stationary individuals."

[0018] S102: Collecting sounds in the target scene through a sound sensor, and determining whether the collected sounds are valid sounds.

[0019] In this embodiment, a preset sound decibel value is set to 55dB, and the sound in the target scene is continuously collected by the sound sensor. The collected sound decibel value is compared with the preset sound decibel value to determine whether the sound decibel value collected by the sound sensor is a valid sound. Specifically, if the sound decibel value collected by the sound sensor is greater than the preset sound decibel value, it is determined to be a valid sound; if the sound decibel value collected by the sound sensor is less than the preset sound decibel value, it is determined to be an invalid sound.

[0020] In this embodiment, the sampling period can also be set to 300ms. If the sound decibel values ​​collected by the sound sensor within three consecutive sampling periods are all greater than the preset sound decibel value, the sound is determined to be valid. If the sound decibel values ​​collected by the sound sensor within three consecutive sampling periods are all less than the preset sound decibel value, or only a single sound decibel value collected is greater than the preset sound decibel value, the sound is determined to be invalid, thereby avoiding misclassifying occasional noise as valid sound. This embodiment uses the sound decibel values ​​collected within three consecutive sampling periods to filter out transient interference.

[0021] For example, a sound sensor collects sound signals from the conference room every 300ms and converts them into decibel values. If the decibel values ​​collected by the sound sensor in the conference room are greater than 55dB for three consecutive sampling periods, and no single decibel value exceeds 100dB, the sound is considered valid, preventing interference from sudden noise caused by objects hitting or falling. If the sound sensor collects decibel values ​​of 56dB, 52dB, and 53dB in three consecutive sampling periods, meaning only one decibel value is greater than 55dB and the other two decibel values ​​are both less than 55dB, the sound is considered invalid.

[0022] S103: If the sound collected by the sound sensor in the target scene is a valid sound, the movement trend range of the people in the target scene is determined according to the image sensor and the microwave radar sensor; if the movement trend range is less than or equal to the first preset range, it is determined that the people in the target scene are in a stationary state, and a projection screen start instruction is sent to the display screen controller through the microwave radar sensor to control the display screen to turn on; if the movement trend range is greater than the first preset range, it is determined that the people in the target scene are in an active state, and the microwave radar sensor sends a projection screen standby instruction to the display screen controller.

[0023] In this embodiment, the first preset range is a rectangular range of 0.2 meters by 0.2 meters. When the microwave radar sensor detects the presence of a person in the target scene and the sound sensor collects valid sound, the image sensor and the microwave radar sensor determine the movement trend range of the person in the target scene, and the movement state of the person in the target scene is determined based on the movement trend range and the first preset range. The display screen is turned on or off according to the movement state of the person in the target scene. The specific control method is as follows: if the movement trend range is less than or equal to the first preset range, the person in the target scene is determined to be stationary, and the microwave radar sensor sends a screen projection start instruction to the display screen controller to control the display screen to turn on; if the movement trend range is greater than the first preset range, the person in the target scene is determined to be active, and the microwave radar sensor sends a screen projection standby instruction to the display screen controller.

[0024] Exemplarily, the coordinates of the people in the conference room are collected by an image sensor to determine the movement status of the people in the conference room, and the echo signals of the people in the conference room are received by a microwave radar sensor. The movement trend range of the people in the conference room is determined based on the historical data of the movement status of the people in the conference room and the historical echo signals of the people in the conference room received by the microwave radar sensor. If the movement trend range of the people in the conference room is less than or equal to the first preset range, it is determined that the people in the conference room are in a stationary state. At this time, a screen projection start instruction is sent to the display screen controller through the microwave radar sensor, thereby controlling the display screen to turn on. If the movement trend range of the people in the conference room is greater than the first preset range, it is determined that the people in the conference room are in an active state. At this time, a screen projection standby instruction is sent to the display screen controller through the microwave radar sensor, waiting for the screen projection to turn on. After 10 seconds, the movement trend range of the people in the conference room is determined again by the image sensor and the microwave radar sensor, and it is determined whether to send a screen projection start instruction through the microwave radar sensor based on the movement trend range of the people in the conference room.

[0025] As can be seen from the above, the embodiments of this application achieve intelligent linkage control in the target scene through the coordinated linkage of microwave radar sensors, sound sensors, and image sensors. The microwave radar sensor analyzes the amplitude change of the echo signal to accurately determine whether there are people in the target scene, effectively eliminating environmental interference. The control command that triggers the display screen to turn on is determined based on the effective sound and the status of the people in the target scene, achieving intelligent linkage control. Through the coordinated linkage of multiple sensors, the error rate is reduced and the anti-interference capability is enhanced.

[0026] In one embodiment of the present application, determining the movement trend range of a person in a target scene using an image sensor and a microwave radar sensor includes: Determine the motion state of the target person based on a determination result of a position change of the person in the target scene detected by the image sensor within a first preset time and a preset position change threshold; If the position change of the person in the target scene is greater than or equal to the preset position change threshold, it is determined that the person in the target scene is in an active state; based on the activity state of the person in the target scene, the movement trend of the person in the target scene is predicted, and the movement trend is uploaded to the microwave radar sensor; The microwave radar sensor receives a preset echo signal and determines the movement trend range of people in the target scene according to the preset echo signal.

[0027] In this embodiment, the first preset time is 5 seconds, and the preset angle threshold is 20°. The image sensor collects the position change of the person in the target scene within the first preset time, and the motion state of the person in the target scene is judged based on the position change and the preset position change threshold. If the position change of the person in the target scene is greater than or equal to the preset position change threshold, it is determined that the person in the target scene is in an active state. The position change is collected by the image sensor for the target image, and the initial position and final position of the person in the target scene are determined based on the target image. The position change of the person in the target scene is determined based on the initial position and the final position; the motion trend of the person in the target scene is predicted based on the activity state of the person in the target scene, and the motion trend is uploaded to the microwave radar sensor; based on the historical data of the posture angle change of the person in the target scene collected by the image sensor and the posture change angle of the person in the target scene collected within the first preset time, the possible posture change of the person in the target scene in the future time is predicted, the microwave radar sensor determines a preset echo signal based on the possible posture change in the future time, and the motion trend range of the person in the target scene is determined based on the preset echo signal.

[0028] Exemplarily, the image data collected by the image sensor within the first preset time is used to determine whether there is a position change of person A in the conference room, and the position change amount of person A is determined based on the position coordinates of person A in the two images collected by the image sensor. For example, within the first preset time of 5 seconds, the image sensor detects that the initial position of person A is (200 pixels, 500 pixels) and the final target is (800 pixels, 500 pixels). At this time, the position change amount of person A is 600 pixels, which is greater than the preset position change threshold of 300 pixels. At this time, it is determined that person A is in an active state, and the image sensor is used to predict the movement trend of person A based on the activity state of person A. , and upload the movement trend of person A to the microwave radar sensor; after receiving the movement trend data, the microwave radar sensor builds a prediction model based on the human body movement characteristics, simulates the position of the reflected echo signal at different times according to the speed and angle of the standing up phase, and receives the predicted amplitude of the echo signal; after completing the standing up action 2 seconds later, the microwave radar sensor determines the predicted amplitude of the echo signal based on the predicted standing up position; after entering the moving phase, the walking movement trend of person A is updated through the image sensor, and the microwave radar sensor predicts the fluctuation range of the echo signal based on the walking movement trend of person A, thereby determining the movement trend range of person A.

[0029] In one embodiment of the present application, determining a preset echo signal according to a microwave radar sensor, and determining a movement trend range of a person in a target scene according to the preset echo signal includes: Determine the reflected wave variation according to the preset echo signal, and determine the phase difference of the reflected wave according to the reflected wave variation; The change in distance between a person in the target scene and the microwave radar sensor is determined by the phase difference of the reflected wave of the microwave radar sensor; Decompose the X-axis distance change and Y-axis distance change based on the distance change between the person and the microwave radar sensor in the target scene; According to the X-axis distance variation, an X-axis distance variation sequence is obtained, and the maximum movement distance of the X-axis is obtained according to the X-axis distance variation sequence; according to the Y-axis distance variation, a Y-axis distance variation sequence is obtained, and the maximum movement distance of the Y-axis is obtained according to the Y-axis distance variation sequence; The movement trend range is calculated based on the maximum moving distance of the X axis and the maximum moving distance of the Y axis.

[0030] In this embodiment, a change in the reflected wave and a phase difference of a preset echo signal are determined, and a change in the distance between a person in the target scene and the microwave radar sensor is calculated based on the relationship between the phase difference and the distance. The distance change is obtained by determining the initial distance and the distance after the movement of the person in the target scene. The distance change is decomposed into an X-axis distance change and a Y-axis distance change. The X-axis distance change and the Y-axis distance change are recorded every 0.5 seconds, and corresponding X-axis distance change sequences and Y-axis distance change sequences are obtained based on the recorded X-axis distance change and Y-axis distance change sequences. The maximum X-axis movement distance and the maximum Y-axis movement distance are obtained based on the X-axis distance change sequence and the Y-axis distance change sequence, respectively. The motion trend range is calculated based on the maximum X-axis movement distance and the maximum Y-axis movement distance.

[0031] For example, the distance change between the person in the target scene and the microwave radar sensor is calculated based on the relationship between the phase difference and the distance of the preset echo signal. If the straight-line distance between person A and the microwave radar sensor is 4 meters before he gets up to change his seat, and the distance between person A and the microwave radar sensor is 4.1 meters after he moves, the distance change is 0.1 meter. The distance change is decomposed into the X-axis distance change and the Y-axis distance change. After the decomposition, the distance change of the X-axis accounts for 60% of the total change, that is, 0.1 meter × 60% = 0.06 meter, and the distance change of the Y-axis accounts for 0.1 meter. 40% of the total change, that is, 0.1×40%=0.04 meters; the microwave radar sensor records the distance change between the X-axis and the Y-axis every 0.5 seconds to form an X-axis distance change sequence and a Y-axis distance change sequence, respectively. The motion trend range is calculated based on the maximum moving distance in the X-axis distance change sequence and the Y-axis distance change sequence. In this example, the X-axis distance change sequence and the Y-axis distance change sequence are 0.25 meters and 0.3 meters, respectively, thereby determining that the motion trend range is 0.075 square meters, which is larger than the first preset range of 0.04 square meters.

[0032] In this embodiment, if the range of movement of a person in the target scene is irregular, an image sensor is used to capture the edge points of the person's movement to form a person's activity profile. A microwave radar sensor is used to convert the pixel coordinates corresponding to the profile extracted by the image sensor into physical coordinates. The microwave radar sensor is used to calibrate the echo signal of each edge point. If the amplitude change of the echo signal of any edge point is less than a preset threshold, the echo signal of each point is determined to be outside the effective activity range and is eliminated. Edge points with an amplitude change greater than the preset threshold are retained. The convex hull of the calibrated edge points is calculated, and the outermost points are connected to form a minimum convex polygon. This polygon is the boundary of the irregular activity range of the person in the target scene, that is, the movement trend range of the person in the target scene. This embodiment, based on the coordinated linkage of the image sensor and the microwave radar sensor, realizes full-process monitoring from contour capture, boundary calibration, to movement trend range determination, ensuring the authenticity and accuracy of the movement trend range.

[0033] In one embodiment of the present application, if the decibel value of the sound collected by the sound sensor is greater than a preset decibel value, it is determined to be a valid sound, including: Use at least one of the following methods to determine whether the sound collected by the sound sensor is valid sound; If the decibel value of each sound collected by the sound sensor within a preset number of consecutive times is greater than the preset decibel value, it is determined to be a valid sound; If each sound decibel value collected by the sound sensor within the second preset time is greater than the preset sound decibel value, it is determined to be a valid sound.

[0034] In this embodiment, the preset number of times is set to 3, the sampling interval is 100ms, the second preset time is 5 seconds, and the preset sound decibel value is 55dB. The sound signals collected by the sound sensor within three consecutive sampling intervals are converted into sound decibel values, and the sound decibel values ​​are compared with the preset sound decibel values. If the sound decibel values ​​collected by the sound sensor three times are 58dB, 62dB, and 60dB, respectively, all greater than the preset sound decibel value of 55dB, then the sound is determined to be valid. The sound signals collected by the sound sensor within the second preset time are converted into sound decibel values, and the sound decibel values ​​are compared with the preset sound decibel value. If each sound decibel value collected within the second preset time is greater than the preset sound decibel value, then the sound is determined to be valid.

[0035] In one embodiment of the present application, collecting sound in a target scene by a sound sensor and determining whether the collected sound is a valid sound further include: Matching the coordinates corresponding to the effective sound collected by the sound sensor with the actual coordinates corresponding to the effective sound collected by the image sensor to obtain a spatiotemporal matching degree; Determine whether the valid sound collected by the sound sensor is an interference sound based on the time-space matching degree; If the spatiotemporal matching degree is less than the preset matching degree, it is determined that the valid sound collected by the sound sensor is an interference sound, and the interference sound is eliminated.

[0036] In this embodiment, a preset matching degree is set to 75%. A sound sensor is used to collect valid sounds within the target scene. The sound source coordinates of the valid sounds are determined using sound arrival time difference localization technology. Simultaneously, an image sensor is used to identify the pixel coordinates of the speaker, which are converted into physical coordinates to obtain the actual sound source coordinates. The coordinate deviation is calculated using the distance formula between two points, and the spatial matching degree is calculated based on the coordinate deviation. The temporal matching degree is determined based on the moment the sound sensor detects the valid sound and the moment the image sensor captures the speaker's mouth opening. The spatiotemporal matching degree is calculated based on the average of the spatial matching degree and the temporal matching degree. The spatiotemporal matching degree is used to determine whether the valid sound collected by the sound sensor is an interfering sound. If the spatiotemporal matching degree is less than the preset matching degree, the valid sound collected by the sound sensor is determined to be an interfering sound and is discarded. If the spatiotemporal matching degree is greater than the preset matching degree, the valid sound collected by the sound sensor is determined to be a true valid sound.

[0037] For example, if the distance between the sound source coordinates and the actual sound source coordinates is 0.14 meters, and the preset maximum allowable deviation is 0.6 meters, the spatial matching degree is (1-0.14 / 0.6) × 100% ≈ 77%. If the difference between the time when the sound sensor detects the valid sound and the time when the image sensor captures the speaker's mouth opening is 0.03 seconds, which is less than the allowed time error of 0.1 seconds, the temporal matching degree at this time is considered 100%. At this time, the spatiotemporal matching degree is 88%, which is greater than the preset matching degree, and the valid sound collected by the sound sensor is a true valid sound. If the sound sensor detects a car horn outside the conference room with a decibel value of 65dB, the sound is determined to be valid. The sound source coordinates of the valid sound and the actual sound source coordinates are detected and calculated by the sound sensor and image sensor respectively, and the spatiotemporal matching degree is 25%, which is less than the preset matching degree, and the car horn is eliminated as an interference sound.

[0038] In one embodiment of the present application, if the motion trend range is greater than a first preset range, it is determined that the person in the target scene is in an active state, and after the microwave radar sensor sends a projection standby instruction to the display controller, the method further includes: After the third preset time, the microwave radar sensor calculates the movement trend range of people in the target scene, and determines whether the microwave radar sensor sends a projection start instruction to the display screen based on the movement trend range.

[0039] In this embodiment, the third preset time is 20 seconds. If the microwave radar sensor sends a screen projection standby command to the display screen controller, the microwave radar sensor will calculate the movement trend range of the people in the target scene again after 20 seconds. If the movement trend range is less than or equal to the first preset range, the microwave radar sensor will send a screen projection start command to the display screen controller to control the display screen to turn on; if the movement trend range is greater than the first preset range, the microwave radar sensor will send a screen projection standby command to the display screen controller and will not turn on the display screen.

[0040] In one embodiment of the present application, after sending a screen projection start instruction to the display screen controller through the microwave radar sensor to control the display screen to turn on, the method further includes: If the microwave radar sensor does not detect an echo signal within the third consecutive preset time, or the sound decibel value collected by the sound sensor is less than the preset sound decibel value, a screen projection off command is sent to the display controller through the microwave radar sensor to control the display screen to turn off.

[0041] In this embodiment, the third preset time is 5 minutes. After the people in the target scene leave, if the microwave radar sensor does not detect an echo signal within the third preset time or the amplitude change of the detected echo signal is less than the preset amplitude change threshold, the microwave radar sensor sends a screen projection closing instruction to the display controller. If some people leave and some people still exist in the target scene, the sound signal in the target scene is collected by the sound sensor, and the sound signal is converted into a sound decibel value. If the sound decibel value is less than the preset sound decibel value, the microwave radar sensor sends a screen projection closing instruction to the display controller. This embodiment sets a third preset time to avoid turning off the display screen when there is a brief silence in the meeting. This embodiment uses the microwave radar sensor and the sound sensor in combination to dually determine whether the display screen is turned on or off, accurately determine the usage status of the display screen, and avoid waste of resources.

[0042] Corresponding to a linkage control method of the above embodiment, Figure 2 This is a structural block diagram of a linkage control system provided by an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown. Figure 2 A linkage control system 20 includes: a microwave signal acquisition module 21, a sound judgment module 22 and a command control module 23.

[0043] The microwave signal acquisition module 21 is configured to acquire the echo signal reflected from the target scene in real time by the microwave radar sensor. If the amplitude change of the echo signal acquired twice consecutively within a first acquisition time interval is greater than or equal to a preset amplitude change threshold, it is determined that there is a person in the target scene. If the amplitude change of the echo signal acquired within the first acquisition time interval is less than the preset amplitude change threshold, it is determined that there is no person in the target scene. The preset amplitude change threshold is a threshold for distinguishing between human interference and environmental interference. A sound determination module 22 is configured to collect sounds in a target scene through a sound sensor and determine whether the collected sounds are valid sounds; The command control module 23 is used to collect the sound in the target scene as valid sound for the sound sensor, and determine the movement trend range of the people in the target scene based on the image sensor and the microwave radar sensor; if the movement trend range is less than or equal to the first preset range, it is determined that the people in the target scene are in a stationary state, and a projection start instruction is sent to the display screen controller through the microwave radar sensor to control the display screen to turn on; if the movement trend range is greater than the first preset range, it is determined that the people in the target scene are in an active state, and the microwave radar sensor sends a projection standby instruction to the display screen controller.

[0044] In one embodiment of the present application, the instruction control module 23 is further configured to determine the motion state of the target person based on a determination result between a position change of the person in the target scene detected by the image sensor within a first preset time and a preset position change threshold; If the position change of the person in the target scene is greater than or equal to the preset position change threshold, it is determined that the person in the target scene is in an active state; based on the activity state of the person in the target scene, the movement trend of the person in the target scene is predicted, and the movement trend is uploaded to the microwave radar sensor; The microwave radar sensor receives a preset echo signal and determines the movement trend range of people in the target scene according to the preset echo signal.

[0045] In one embodiment of the present application, the instruction control module 23 is further configured to determine a change in the reflected wave according to a preset echo signal, and determine a phase difference of the reflected wave according to the change in the reflected wave; The change in distance between a person in the target scene and the microwave radar sensor is determined by the phase difference of the reflected wave of the microwave radar sensor; Decompose the distance change between the person and the microwave radar sensor in the target scene into the X-axis distance change and the Y-axis distance change, and calculate the motion trend range based on the X-axis distance change and the Y-axis distance change; Obtain an X-axis distance variation sequence according to the X-axis distance variation, and obtain the maximum movement distance of the X-axis according to the X-axis distance variation sequence; Obtain a Y-axis distance variation sequence according to the Y-axis distance variation, and obtain a maximum Y-axis movement distance according to the Y-axis distance variation sequence; The movement trend range is calculated based on the maximum moving distance of the X axis and the maximum moving distance of the Y axis.

[0046] In one embodiment of the present application, the sound control module 22 is configured to determine whether the sound collected by the sound sensor is a valid sound by using at least one of the following methods: If the decibel value of each sound collected by the sound sensor within a preset number of consecutive times is greater than the preset decibel value, it is determined to be a valid sound; If each sound decibel value collected by the sound sensor within the second preset time is greater than the preset sound decibel value, it is determined to be a valid sound.

[0047] In one embodiment of the present application, the sound control module 22 is further configured to match the coordinates corresponding to the valid sound collected by the sound sensor with the actual coordinates corresponding to the valid sound collected by the image sensor to obtain a spatiotemporal matching degree; Determine whether the valid sound collected by the sound sensor is an interference sound based on the time-space matching degree; If the spatiotemporal matching degree is less than the preset matching degree, it is determined that the valid sound collected by the sound sensor is an interference sound, and the interference sound is eliminated.

[0048] In one embodiment of the present application, the command control module 23 is also used to calculate the movement trend range of people in the target scene through the microwave radar sensor after the third preset time, and determine whether the microwave radar sensor sends a projection start command to the display screen based on the movement trend range.

[0049] See also Figure 3 , Figure 3 This is a schematic block diagram of an electronic device provided in one embodiment of the present application. Figure 3 The electronic device 300 in the embodiment shown may include: one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memory 304 is used to store computer programs, which include program instructions. The processor 301 is used to execute the program instructions stored in the memory 304. The processor 301 is configured to call the program instructions to execute the functions of the modules / units in the above-mentioned system embodiments, such as Figure 2 The functions of the microwave signal acquisition module 21, the sound judgment module 22 and the instruction control module 23 are shown.

[0050] It should be understood that in the embodiment of the present application, the processor 301 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0051] The input device 302 may include a touchpad, a fingerprint collection sensor (for collecting user fingerprint information and fingerprint direction information), a microphone, etc. The output device 303 may include a display (LCD, etc.), a speaker, etc.

[0052] The memory 304 may include a read-only memory and a random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include a non-volatile random access memory. For example, the memory 304 may also store information such as echo signal data, sound data, and image data.

[0053] In a specific implementation, the processor 301, input device 302, and output device 303 described in the embodiment of the present application can execute the implementation method described in a linkage control method provided in the embodiment of the present application, and can also execute the implementation method of the electronic device described in the embodiment of the present application, which will not be repeated here.

[0054] In another embodiment of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, all or part of the process of the method in the above embodiment is implemented. The computer program can also be used to instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above method embodiments are implemented. The computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can include: any entity or system capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium.

[0055] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the aforementioned embodiments, such as the electronic device's hard drive or memory. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. Furthermore, the computer-readable storage medium can include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or is about to be output.

[0056] Those skilled in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0057] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0058] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of modules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules, units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces or modules, or can be an electrical, mechanical or other form of connection.

[0059] Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules, i.e., they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected based on actual needs to achieve the objectives of the embodiments of the present application.

[0060] In addition, the functional modules in the various embodiments of the present application may be integrated into a single processing unit, or each module may exist physically separately, or two or more modules may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0061] The above are only specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and such modifications or substitutions should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A linkage control method, characterized in that: Applied to a linkage control device, the linkage control device includes a microwave radar sensor, a sound sensor, an image sensor, and a display screen controller, the method includes: Acquire an echo signal reflected from a target scene received in real time by a microwave radar sensor, and determine that a person is present in the target scene if the amplitude change of the echo signal acquired two times consecutively within a first acquisition time interval is greater than or equal to a preset amplitude change threshold; If the amplitude variation of the echo signal collected within the first collection time interval is less than a preset amplitude variation threshold, it is determined that no one is in the target scene, where the preset amplitude variation threshold is a threshold for distinguishing between human body and environmental interference; The sound sensor is used to collect the sound in the target scene and determine whether the collected sound is a valid sound; If the sound collected by the sound sensor in the target scene is a valid sound, the movement trend range of the person in the target scene is determined according to the image sensor and the microwave radar sensor; If the motion trend range is less than or equal to the first preset range, it is determined that the person in the target scene is in a stationary state, and a projection screen start instruction is sent to the display screen controller through the microwave radar sensor to control the display screen to turn on; if the motion trend range is greater than the first preset range, it is determined that the person in the target scene is in an active state, and the microwave radar sensor sends a projection screen standby instruction to the display screen controller.

2. The linkage control method according to claim 1, wherein: Determining the movement trend range of a person in a target scene based on an image sensor and a microwave radar sensor includes: Determine the motion state of the target person based on a determination result of a position change of the person in the target scene detected by the image sensor within a first preset time and a preset position change threshold; If the position change of the person in the target scene is greater than or equal to a preset position change threshold, it is determined that the person in the target scene is in an active state; based on the activity state of the person in the target scene, the movement trend of the person in the target scene is predicted, and the movement trend is uploaded to the microwave radar sensor; Based on the microwave radar sensor receiving a preset echo signal, the movement trend range of the person in the target scene is determined according to the preset echo signal.

3. The linkage control method according to claim 2, wherein: The receiving a preset echo signal based on the microwave radar sensor and determining the movement trend range of a person in the target scene according to the preset echo signal includes: Determining a change in the reflected wave according to the preset echo signal, and determining a phase difference of the reflected wave according to the change in the reflected wave; Determining a change in the distance between a person in the target scene and the microwave radar sensor by using a phase difference of a reflected wave from the microwave radar sensor; An X-axis distance change and a Y-axis distance change are decomposed according to a distance change between a person in the target scene and the microwave radar sensor, and the motion trend range is calculated according to the X-axis distance change and the Y-axis distance change.

4. The linkage control method according to claim 3, wherein: Decomposing an X-axis distance change and a Y-axis distance change according to a distance change between a person in the target scene and the microwave radar sensor, and calculating the motion trend range according to the X-axis distance change and the Y-axis distance change, includes: Obtain an X-axis distance variation sequence according to the X-axis distance variation, and obtain a maximum X-axis movement distance according to the X-axis distance variation sequence; Obtain a Y-axis distance variation sequence according to the Y-axis distance variation, and obtain a maximum movement distance of the Y-axis according to the Y-axis distance variation sequence; The movement trend range is calculated according to the maximum movement distance of the X axis and the maximum movement distance of the Y axis.

5. The linkage control method according to claim 1, wherein: The collecting of sounds in the target scene by the sound sensor and determining whether the collected sounds are valid sounds include: Determining whether the sound collected by the sound sensor is a valid sound by using at least one of the following methods: If the decibel value of each sound collected by the sound sensor within a preset number of consecutive times is greater than the preset decibel value, it is determined to be a valid sound; If each sound decibel value collected by the sound sensor within the second preset time is greater than the preset sound decibel value, it is determined to be a valid sound.

6. The linkage control method according to claim 1, wherein: The step of collecting the sound in the target scene by the sound sensor and determining whether the collected sound is a valid sound further includes: Matching the coordinates corresponding to the valid sound collected by the sound sensor with the actual coordinates corresponding to the valid sound collected by the image sensor to obtain a spatiotemporal matching degree; Determining whether the valid sound collected by the sound sensor is an interference sound based on the spatiotemporal matching degree; If the spatiotemporal matching degree is less than a preset matching degree, it is determined that the valid sound collected by the sound sensor is an interference sound, and the interference sound is eliminated.

7. The linkage control method according to claim 1, wherein: After the microwave radar sensor sends a screen projection standby instruction to the display screen controller, the method further includes: After the third preset time, the microwave radar sensor calculates the movement trend range of the people in the target scene, and determines whether the microwave radar sensor sends a screen projection start instruction to the display screen based on the movement trend range.

8. A linkage control system, characterized in that: include: A microwave signal acquisition module is configured to acquire, in real time, an echo signal reflected from a target scene received by a microwave radar sensor. If the amplitude variation of the echo signals acquired twice consecutively within a first acquisition time interval is greater than or equal to a preset amplitude variation threshold, it is determined that a person is present in the target scene. If the amplitude variation of the echo signals acquired within the first acquisition time interval is less than the preset amplitude variation threshold, it is determined that no one is present in the target scene. The preset amplitude variation threshold is a threshold for distinguishing between human interference and environmental interference. A sound judgment module is used to collect the sound in the target scene through the sound sensor and determine whether the collected sound is valid sound; The command control module is used to collect the sound in the target scene as valid sound based on the sound sensor, and determine the movement trend range of the people in the target scene based on the image sensor and the microwave radar sensor; if the movement trend range is less than or equal to the first preset range, it is determined that the people in the target scene are in a stationary state, and a projection screen start instruction is sent to the display screen controller through the microwave radar sensor to control the display screen to turn on; if the movement trend range is greater than the first preset range, it is determined that the people in the target scene are in an active state, and the microwave radar sensor sends a projection screen standby instruction to the display screen controller.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images