Image information acquisition method and device, storage medium and electronic device

Abstract

The invention discloses an image information acquisition method and device, a storage medium and an electronic device, and the method comprises the steps: controlling a target device to transmit a millimeter wave to a target region, and determining a first object in the target region according to a reflected millimeter wave corresponding to the millimeter wave; performing three-dimensional reconstruction on the target area, and screening the first objects according to the target area after the three-dimensional reconstruction so as to determine target objects in the target area; and controlling a dual-light holder on the target equipment to shoot the target object so as to obtain image information of the target object. By means of the method, the problem that missed detection and false detection of personnel in the target area possibly exist when the target area is shot through an infrared light camera and a visible light camera in the related technology can be solved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular, to a method and device for acquiring image information, a storage medium, and an electronic device. Background Art

[0002] Currently, the use of robots for security patrols has become a new trend. It has begun to be applied in fields including electricity, prisons, and banks. Usually, the robot uses the visible light camera and infrared camera on board to take pictures and observe the surrounding environment while traveling along a specific route, and then uses image recognition technology to make abnormal judgments.

[0003] However, visible light cameras may have the problem of false detection and missed detection of moving objects, especially the problem of missed detection of people in bad weather such as wind and rain; at the same time, infrared cameras have fewer image texture features and are more difficult to recognize, which makes it difficult to effectively support related abnormality recognition applications and can only be used as an auxiliary recognition method.

[0004] There is no effective solution yet for the problem of missed detection or false detection of people in the target area when shooting the target area with infrared cameras and visible light cameras in the related technology. Summary of the invention

[0005] The embodiments of the present application relate to a method and device for acquiring image information, a storage medium and an electronic device. Through the embodiments of the present application, the problem of missed detection or false detection of people in the target area caused by shooting the target area with an infrared camera and a visible light camera in the related technology can be solved.

[0006] According to an embodiment of the embodiments of the present application, a method for acquiring image information is provided, including: controlling a target device to send millimeter waves to a target area, and determining a first object in the target area based on a reflected millimeter wave corresponding to the millimeter wave; performing three-dimensional reconstruction of the target area, and screening the first object based on the three-dimensionally reconstructed target area to determine the target object in the target area; controlling a dual-light gimbal on the target device to shoot the target object to acquire image information of the target object.

[0007] In an exemplary embodiment, performing three-dimensional reconstruction of the target area includes: determining motion state information of the first object based on the reflected millimeter wave, wherein the motion state information includes at least one of the following: a distance of the first object relative to the target device, a speed of the first object, an acceleration of the target object, and an azimuth of the target object relative to the target device; performing three-dimensional reconstruction of the target area based on the motion state information, wherein each first object is each three-dimensional feature point in the target area after three-dimensional reconstruction.

[0008] In an exemplary embodiment, the first objects are screened according to the three-dimensionally reconstructed target area to determine the target object within the target area, including: determining the curvature change value and the height of each first object according to the three-dimensional feature points of each first object, and obtaining the speed of each first object; and screening the first objects according to the curvature change value, the height and the speed to determine the target object.

[0009] In an exemplary embodiment, the first object is screened according to the curvature change value, the height and the speed to determine the target object, including: screening out a second object whose curvature change value is less than or equal to a preset curvature change value from the first object according to the curvature change value; screening out a third object whose height is less than or equal to a preset height from the second object according to the height; and screening out a target object whose speed is less than or equal to a preset speed from the third object according to the speed.

[0010] In an exemplary embodiment, controlling the dual-light gimbal on the target device to shoot the target area includes: determining a first angle of the target object relative to the target device, and determining a distance of the target object relative to the target device; adjusting a first shooting direction of the dual-light gimbal according to the first angle, and when determining that the distance is less than or equal to a preset distance, adjusting a first focal length of the dual-light gimbal according to the angle and the distance; and controlling the dual-light gimbal to shoot the target object based on the first shooting direction and the first focal length.

[0011] In an exemplary embodiment, after determining the distance of the target object relative to the target device, the method also includes: controlling the target device to move toward the target object when it is determined that the distance is greater than the preset distance; determining the real-time distance between the target device and the target object when it is determined that the moving distance of the target device is less than or equal to the maximum moving distance of the target device; determining a second angle of the target object relative to the target device when it is determined that the real-time distance is less than or equal to the preset distance; adjusting a second shooting direction of the dual-light gimbal according to the second angle, and adjusting a second focal length of the dual-light gimbal according to the second shooting direction and the real-time distance; and controlling the dual-light gimbal to shoot the target device based on the second shooting direction and the second focal length.

[0012] In an exemplary embodiment, after acquiring the image information of the target object, the method further includes: judging the image information according to a dual-light fusion algorithm; and generating alarm information when the judgment result is consistent with a preset judgment result.

[0013] According to another embodiment of the embodiment of the present application, there is also provided an image information acquisition device, including: a first control module, used to control the target device to send millimeter waves to a target area, and determine a first object in the target area based on a reflected millimeter wave corresponding to the millimeter wave; a screening module, used to perform three-dimensional reconstruction of the target area, and screen the first object based on the three-dimensionally reconstructed target area to determine the target object in the target area; a second control module, used to control the dual-light gimbal on the target device to shoot the target object to obtain image information of the target object.

[0014] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above method when running.

[0015] According to another aspect of the embodiments of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method through the computer program.

[0016] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, wherein the computer program is executed by a processor to perform the method.

[0017] In an embodiment of the present application, the target device is controlled to send millimeter waves to the target area, and the first object in the target area is determined according to the reflected millimeter waves corresponding to the millimeter waves; the target area is reconstructed in three dimensions, and the first object is screened according to the target area after three-dimensional reconstruction to determine the target object in the target area; the dual-light pan-tilt in the target device is controlled to shoot the target object to obtain image information of the target object. That is to say, the embodiment of the present application screens the first object in the target area by millimeter wave technology, and shoots the screened target object by the dual-light pan-tilt. Through the embodiment of the present application, the problem of missed detection and false detection of personnel in the target area caused by shooting the target area by infrared light camera and visible light camera in the related technology can be solved, all the first objects in the target area are detected by millimeter wave technology, the missed detection rate of the first object in the target area is reduced, and the first object is screened by millimeter wave technology, and the target object is reconfirmed by the dual-light pan-tilt to avoid false detection of the target object in the target area. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 It is a hardware structure block diagram of a computer terminal of a method for acquiring image information in an embodiment of the present application;

[0020] Figure 2 is a flowchart of a method for acquiring image information according to an embodiment of the present application;

[0021] Figure 3 is a flow chart of a security monitoring method based on the combination of millimeter wave radar and dual light camera according to an optional embodiment of the present application;

[0022] Figure 4 It is a structural block diagram of a device for acquiring image information according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices; "plurality" means two or more.

[0025] The method embodiments provided in the embodiments of the present application can be executed in a computer terminal or similar computing device or cloud platform or independent physical server or software platform, wherein the above software platform is run through one or more servers. Taking running on a computer terminal as an example, Figure 1 1 is a hardware structure block diagram of a computer terminal of a method for acquiring image information in an embodiment of the present application. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data. In an exemplary embodiment, the computer terminal may also include a transmission device 106 for communication functions and an input and output device 108, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. A person skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Equivalent functions or comparisons shown Figure 1 A different configuration with more features is shown.

[0026] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as computer programs corresponding to the methods in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, that is, the above-mentioned methods are implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0027] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0028] In this embodiment, a method for acquiring image information is provided, which is applied to the above-mentioned computer terminal. Figure 2 : is a flowchart of a method for acquiring image information according to an embodiment of the present application, the process includes the following steps:

[0029] Step S202, controlling the target device to send millimeter waves to a target area, and determining a first object in the target area according to a reflected millimeter wave corresponding to the millimeter wave;

[0030] The target device may be a robot equipped with a millimeter wave radar and a dual-light gimbal. The target area may be a sensitive area inside a centralized photovoltaic station or a booster station where personnel are prohibited from entering.

[0031] It can be understood that when an electromagnetic wave emitted at a millimeter wave frequency (i.e., the millimeter wave of the present application) encounters an object (i.e., the first object of the present application), reflection will occur, thereby generating the above-mentioned reflected millimeter wave (i.e., the reflection point corresponding to the millimeter wave is the first object).

[0032] Step S204, performing three-dimensional reconstruction on the target area, and screening the first object according to the three-dimensionally reconstructed target area to determine the target object in the target area;

[0033] The first object may be any movable or immovable object or suspected person in the target area. The target object may be some suspected person in the target area.

[0034] Step S206: controlling the dual-light gimbal on the target device to shoot the target object to obtain image information of the target object.

[0035] In an embodiment of the present application, the target device is controlled to send millimeter waves to the target area, and the first object in the target area is determined according to the reflected millimeter waves corresponding to the millimeter waves; the target area is reconstructed in three dimensions, and the first object is screened according to the target area after three-dimensional reconstruction to determine the target object in the target area; the dual-light pan-tilt in the target device is controlled to shoot the target object to obtain image information of the target object. That is to say, the embodiment of the present application screens the first object in the target area by millimeter wave technology, and shoots the screened target object by the dual-light pan-tilt. Through the embodiment of the present application, the problem of missed detection and false detection of personnel in the target area caused by shooting the target area by infrared light camera and visible light camera in the related technology can be solved, all the first objects in the target area are detected by millimeter wave technology, the missed detection rate of the first object in the target area is reduced, and the first object is screened by millimeter wave technology, and the target object is reconfirmed by the dual-light pan-tilt to avoid false detection of the target object in the target area.

[0036] Optionally, performing three-dimensional reconstruction of the target area in the above step S204 includes: determining motion state information of the first object based on the reflected millimeter wave, wherein the motion state information includes at least one of the following: the distance of the first object relative to the target device, the speed of the first object, the acceleration of the target object and the azimuth of the target object relative to the target device; performing three-dimensional reconstruction of the target area based on the motion state information, wherein each first object is each three-dimensional feature point in the target area after three-dimensional reconstruction.

[0037] It is understandable that it is necessary to obtain the motion state information of the first object. The motion state information may include: the distance and azimuth between the first object and the target device (eg, robot), the speed and acceleration of the first object.

[0038] By analyzing the amplitude and phase changes of the reflected millimeter wave, the distance, speed, acceleration, azimuth and other parameters between the first object and the target device (i.e., the motion state information of the present application) can be calculated. Specifically:

[0039] Distance: By measuring the phase change, the distance between the first object and the target device can be calculated; Speed: The radial speed of the first object can be determined by the Doppler effect, that is, the change in frequency; Acceleration: The acceleration of the first object can be further calculated by analyzing the speed change; Azimuth: The azimuth of the first object can be determined by comparing the signal strength and phase received by different antennas.

[0040] Based on the motion state information, it can be determined whether the first object is in a motion state. If the motion state of the fourth object in the first object is a static state within a preset time period, the fourth object can be marked as a non-suspected human object, that is, the fourth object is not the target object; if the motion state of the fifth object in the first object is a static state and a motion state or only a motion state within the preset time period, the fifth object can be marked as a suspected human object, that is, the fifth object is the target object.

[0041] According to the above technical solution, millimeter wave technology can not only quickly detect moving objects (i.e. target objects) in the target area, but also provide the motion state information of the target objects and perform three-dimensional environment reconstruction. For security monitoring systems, it can greatly improve the efficiency and accuracy of monitoring, especially in bad weather conditions with low visibility, and can maintain stable working performance.

[0042] Optionally, the above-mentioned step S204 filters the first objects according to the target area after three-dimensional reconstruction to determine the target object within the target area, including: determining the curvature change value and the height of each first object according to the three-dimensional feature points of each first object, and obtaining the speed of each first object; filtering the first objects according to the curvature change value, the height and the speed to determine the target object.

[0043] Among them, the first object is screened according to the curvature change value, the height and the speed to determine the target object, including: screening out a second object whose curvature change value is less than or equal to a preset curvature change value from the first object according to the curvature change value; screening out a third object whose height is less than or equal to a preset height from the second object according to the height; and screening out a target object whose speed is less than or equal to a preset speed from the third object according to the speed.

[0044] It is understandable that the target area needs to be obtained after three-dimensional reconstruction first. Millimeter wave technology can obtain the reflected signal of the surrounding environment of the target area, thereby reconstructing a three-dimensional environment model. This model contains relevant information of all reflecting targets (i.e., the first object), including distance, azimuth, altitude and other information.

[0045] After obtaining the target area after 3D reconstruction, the first object needs to be screened. The purpose of the screening process is to find the moving object (i.e., the target object) that is suspected to be a human from the first object. The specific screening steps include:

[0046] Determine the curvature change value: Analyze the three-dimensional feature points of each first object and calculate the change in surface curvature corresponding to each first object. The contour of the human body usually has a relatively uniform curvature change, while the curvature change of other objects (for example, trees, buildings) may be more drastic. By setting a curvature change threshold (i.e., the preset curvature change value of this application), objects suspected to be humans (i.e., the second objects of this application) can be preliminarily screened out.

[0047] Get height: further analyze the height information of each second object. The height of humans is usually within a certain height range (i.e., the preset height of this application), and second objects beyond this height range can be excluded. For example, objects that are too high are unlikely to be humans, and then the third object (i.e., an object with a height less than or equal to the preset height) is determined in the second object.

[0048] Acquisition speed: The target's radial speed can be acquired through the millimeter-wave radar in the target device, and the target's moving speed can be calculated by combining the multi-frame data continuously scanned by the millimeter-wave radar. A speed threshold is set (i.e., the preset speed of this application). Third objects that are too fast may not be human.

[0049] In addition, the third object with too slow speed may not be a human being. Therefore, a minimum speed may be set, and target objects with speeds greater than or equal to the minimum speed and less than or equal to the preset speed may be screened out from the third objects according to the speed.

[0050] Determine the target object: Combine the three indicators of curvature change value, height and speed to comprehensively screen the first object to determine which are the real target objects, that is, moving objects that are suspected to be humans. For example, if an object has uniform curvature change, a height within the normal range for humans, and a moderate moving speed, then it is more likely to be identified as a target object.

[0051] Through such a screening process, false alarms and missed alarms can be effectively reduced, the accuracy of identification can be improved, and the efficient operation of the security monitoring system can be ensured.

[0052] Optionally, the above-mentioned step S206 of controlling the dual-light gimbal on the target device to shoot the target area includes: determining a first angle of the target object relative to the target device, and determining a distance of the target object relative to the target device; adjusting a first shooting direction of the dual-light gimbal according to the first angle, and when determining that the distance is less than or equal to a preset distance, adjusting a first focal length of the dual-light gimbal according to the angle and the distance; and controlling the dual-light gimbal to shoot the target object based on the first shooting direction and the first focal length.

[0053] It is understandable that the specific steps of photographing the target object include:

[0054] 1. Determine a first angle of the target object relative to the target device, and determine a distance of the target object relative to the target device;

[0055] The millimeter wave radar in the target device can be used to obtain the location information of the target object (for example, a suspected person). The millimeter wave radar transmits millimeter waves and receives the reflected signal. According to the time difference and phase change of the signal, the azimuth (i.e., the first angle of this application) and distance of the target object relative to the target device (which can be a robot equipped with a millimeter wave radar) can be calculated. The azimuth provides the relative position of the target object relative to the target device on the horizontal plane, while the distance provides the straight-line distance between the target object and the target device.

[0056] 2. Adjusting a first shooting direction of the dual-light gimbal, and when determining that the distance is less than or equal to a preset distance, adjusting a first focal length of the dual-light gimbal according to the angle and the distance.

[0057] After obtaining the first angle and distance of the target object, the shooting direction of the dual-light gimbal can be automatically adjusted according to the first angle to ensure that the dual-light gimbal is aimed at the target object. At the same time, if the distance between the target object and the target device is less than or equal to the preset distance, the robot will further adjust the focal length of the dual-light gimbal to obtain a clearer shooting effect. The preset distance is set to ensure that the dual-light camera can obtain higher quality images when the target is close enough, thereby improving the accuracy of image recognition.

[0058] 3. Photograph the target object.

[0059] Based on the adjusted first shooting direction and first focal length, the robot controls the dual-light gimbal to shoot the target object. The dual-light gimbal can obtain images under different spectrums, which helps to more accurately identify and analyze the target object in various environments (for example: daytime, night, bad weather, etc.).

[0060] That is, the embodiment of the present application uses the high-precision positioning information of the millimeter-wave radar to intelligently adjust the shooting angle and focal length of the dual-light gimbal to achieve efficient and accurate identification and monitoring of the target object. The above technical solution not only improves the shooting efficiency, but also ensures that clear images can be obtained under different environmental conditions, thereby improving the accuracy and reliability of security monitoring.

[0061] Wherein, after determining the distance of the target object relative to the target device, the method also includes: when it is determined that the distance is greater than the preset distance, controlling the target device to move toward the target object; when it is determined that the moving distance of the target device is less than or equal to the maximum moving distance of the target device, determining the real-time distance between the target device and the target object; when it is determined that the real-time distance is less than or equal to the preset distance, determining a second angle of the target object relative to the target device; adjusting a second shooting direction of the dual-light gimbal according to the second angle, and adjusting a second focal length of the dual-light gimbal according to the second shooting direction and the real-time distance; and controlling the dual-light gimbal to shoot the target device based on the second shooting direction and the second focal length.

[0062] It is understandable that after the millimeter-wave radar is used to identify a suspected human object (i.e., the target object), the distance between the target object and the target device can be calculated. If this distance exceeds the preset distance, it means that the target object is far away from the target device and may be beyond the effective shooting range of the dual-light gimbal. At this time, the target device will start the autonomous navigation function and move in the direction of the target object to shorten the distance between the two, improve the clarity of the shooting and the accuracy of the recognition.

[0063] When the robot (i.e., the target device) moves, the millimeter-wave radar will be used to continuously monitor the distance between the robot and the target object to ensure that the robot does not move too far or too close. If the robot has moved a certain distance but has not reached the maximum moving distance limit (i.e., the maximum moving distance of the target device), the real-time distance between the robot and the target object can be calculated again to ensure that the robot approaches the target object within a safe range.

[0064] When the real-time distance is less than or equal to the preset distance, it means that the robot has approached the distance where it can effectively shoot the target object. At this time, the second shooting angle and second focal length of the dual-light gimbal can be further adjusted based on the real-time distance to obtain the best image quality.

[0065] At this time, the dual-light gimbal can capture the image of the target object under the best shooting conditions for subsequent identification and judgment.

[0066] When the target device has moved the maximum moving distance, but the real-time distance is still greater than or equal to the preset distance, the relevant information of the target object can be sent to the tracking backend system so that the backend system can dispatch staff to the target area for inspection.

[0067] The above technical solution ensures that during the security monitoring process, even if the target object is at a long distance or moving, the system can achieve accurate and clear shooting by adjusting the parameters of the robot and the dual-light gimbal (for example: the shooting angle and focal length of the dual-light gimbal), thereby improving the efficiency and accuracy of identifying abnormal situations.

[0068] Optionally, after acquiring the image information of the target object in step S206, the method further includes: judging the image information according to a dual-light fusion algorithm; and generating alarm information when the judgment result is consistent with a preset judgment result.

[0069] It can be understood that after acquiring the image information of the target object, it is possible to determine whether the target object is a human being based on the clear image information. When the target object is determined to be a human being and is in the target area, an alarm message can be generated to monitor the target area.

[0070] In order to better understand the process of the above-mentioned method for obtaining image information, the above-mentioned method for obtaining image information is described below in combination with an optional embodiment, but it is not used to limit the technical solution of the embodiment of the present application.

[0071] Visible light cameras may have the problem of false detection and missed detection of moving objects, especially the problem of missed detection of people in bad weather such as wind and rain; they may also have low collection efficiency, low collection efficiency for objects with different focal lengths, and inability to achieve synchronous and rapid data collection within a large distance range. At the same time, infrared cameras have fewer image texture features and are difficult to identify. It is difficult to effectively support related abnormality recognition applications and can only be used as an auxiliary recognition method. LiDAR is also expensive.

[0072] Therefore, an optional embodiment of the present application proposes a security monitoring method based on the combination of millimeter wave radar and dual-light camera. Figure 3 is a flow chart of a security monitoring method based on the combination of millimeter wave radar and dual light camera according to an optional embodiment of the present application, such as Figure 3 As shown:

[0073] Step S301, controlling the mobile robot device to inspect a certain area.

[0074] A mobile robot device (i.e. the target device of this application, also referred to as the robot in the following text) is equipped with a millimeter-wave radar and a dual-light gimbal. The robot inspects a certain area (e.g., the interior of a centralized photovoltaic station or a booster station) along a specific route and identifies sensitive areas within the inspection area that are prohibited for entry (i.e., the target area of ​​this application).

[0075] Among them, 4 millimeter-wave radars can be set around the robot, and the 4 millimeter-wave radars are used to scan the 360-degree environment around the robot.

[0076] The robot can be equipped with a laser radar and can inspect a certain area along a specific route. It can also independently determine the inspection route and then realize automatic navigation to inspect the area.

[0077] Step S302: using millimeter wave radar to determine suspected human objects moving around.

[0078] Use millimeter-wave radar to judge suspected human objects moving around. Emits millimeter waves in all directions, reconstructs the position of the reflection point (i.e., the position of the first object) by the amplitude and phase of the reflected millimeter waves, obtains the distance, speed, acceleration, and azimuth of the reflecting target (i.e., the first object), and reconstructs the surrounding environment in three dimensions. Fits the three-dimensional feature points, tracks the moving objects (i.e., the second objects) with uniform curvature changes similar to the human body, and removes objects with sharp surface changes with large curvature changes; evaluates the height of the fitted object, and determines it as non-human if it exceeds a certain threshold (i.e., the preset height of this application); then divides the first object according to a certain speed threshold (i.e., the speed threshold of this application) based on the speed, and determines it as a non-human moving object if it exceeds a certain speed threshold; marks the suspected human object (i.e., the target object).

[0079] Step S303: taking photos of suspected human objects to collect evidence.

[0080] When a suspected human object is detected in a sensitive area, the dual-light gimbal is automatically turned in direction and the focal length (i.e., the first focal length) is adjusted based on the angle (i.e., the first angle) and distance of the detected suspected human object relative to the robot to take photos for evidence collection.

[0081] If a suspected human object is found to be beyond the shooting range of the dual-light gimbal, the captured image is blurred and cannot be identified, and the suspected human object is still moving continuously, the robot will automatically navigate and track it based on the positioning information, shortening the distance between it and the suspected human object.

[0082] If it is found that the moving speed of a suspected human object exceeds the moving distance of the robot, causing the real-time distance to become farther and farther, the relevant information corresponding to the suspected human object will be sent to the background system for prompting. If there are other cameras around the current location of the suspected human object, other cameras can be turned to shoot the suspected human object. If there are no other cameras, the background can dispatch personnel to conduct on-site inspection.

[0083] Step S304: Generate alarm information.

[0084] If a person's image (i.e., image information) can be captured clearly, the captured person's image will be recognized by the robot itself or transmitted to the background system for recognition through a proprietary data link, and the person will be judged through the dual-light fusion algorithm. If it is determined that the person's image contains humans and the humans contained in the person's image are active in sensitive areas, an alarm message will be generated.

[0085] According to the optional embodiments of the present application, the inspection efficiency can be improved, and the millimeter wave radar can quickly scan the moving objects in the surrounding environment, unlike the image inspection mode such as visible light, which requires point-by-point collection and judgment; at the same time, the adaptability is improved, and safety monitoring can be accurately implemented in various adverse windy and rainy weather conditions and night environments.

[0086] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0087] The present application also provides a structural block diagram of a device for acquiring image information. Figure 4 is a structural block diagram of an apparatus for acquiring image information according to an embodiment of the present application; Figure 4 As shown, including:

[0088] A first control module 42, used to control the target device to send millimeter waves to a target area, and determine a first object in the target area according to a reflected millimeter wave corresponding to the millimeter wave;

[0089] A screening module 44 is used to perform three-dimensional reconstruction on the target area, and screen the first object according to the three-dimensionally reconstructed target area to determine the target object in the target area;

[0090] The second control module 46 is used to control the dual-light pan / tilt platform on the target device to shoot the target object to obtain image information of the target object.

[0091] Through the above-mentioned device, the target device is controlled to send millimeter waves to the target area, and the first object in the target area is determined according to the reflected millimeter waves corresponding to the millimeter waves; the target area is reconstructed in three dimensions, and the first object is screened according to the target area after three-dimensional reconstruction to determine the target object in the target area; the dual-light pan-tilt in the target device is controlled to shoot the target object to obtain image information of the target object. That is to say, the embodiment of the present application screens the first object in the target area through millimeter wave technology, and shoots the screened target object through the dual-light pan-tilt. Through the embodiment of the present application, the problem of missed detection and false detection of personnel in the target area caused by shooting the target area through infrared light cameras and visible light cameras in the related technology can be solved, all first objects in the target area are detected through millimeter wave technology, the missed detection rate of the first object in the target area is reduced, and the first object is screened through millimeter wave technology, and the target object is reconfirmed through the dual-light pan-tilt to avoid false detection of the target object in the target area.

[0092] In an exemplary embodiment, the screening module 44 is also used to determine the motion state information of the first object based on the reflected millimeter wave, wherein the motion state information includes at least one of the following: the distance of the first object relative to the target device, the speed of the first object, the acceleration of the target object and the azimuth of the target object relative to the target device; and perform three-dimensional reconstruction of the target area based on the motion state information, wherein each first object is each three-dimensional feature point in the target area after three-dimensional reconstruction.

[0093] In an exemplary embodiment, the screening module 44 is also used to determine the curvature change value and height of each first object based on the three-dimensional feature points of each first object, and obtain the speed of each first object; the first objects are screened according to the curvature change value, the height and the speed to determine the target object.

[0094] In an exemplary embodiment, the screening module 44 is also used to screen out a second object whose curvature change value is less than or equal to a preset curvature change value in the first object according to the curvature change value; to screen out a third object whose height is less than or equal to a preset height in the second object according to the height; and to screen out a target object whose speed is less than or equal to a preset speed in the third object according to the speed.

[0095] In an exemplary embodiment, the second control module is also used to determine a first angle of the target object relative to the target device, and to determine a distance of the target object relative to the target device; adjust a first shooting direction of the dual-light gimbal according to the first angle, and when it is determined that the distance is less than or equal to a preset distance, adjust a first focal length of the dual-light gimbal according to the angle and the distance; and control the dual-light gimbal to shoot the target object based on the first shooting direction and the first focal length.

[0096] In an exemplary embodiment, the second control module is further used to control the target device to move toward the target object when it is determined that the distance is greater than the preset distance; determine the real-time distance between the target device and the target object when it is determined that the moving distance of the target device is less than or equal to the maximum moving distance of the target device; determine the second angle of the target object relative to the target device when it is determined that the real-time distance is less than or equal to the preset distance; adjust the second shooting direction of the dual-light gimbal according to the second angle, and adjust the second focal length of the dual-light gimbal according to the second shooting direction and the real-time distance; and control the dual-light gimbal to shoot the target device based on the second shooting direction and the second focal length.

[0097] In an exemplary embodiment, the second control module is further used to judge the image information according to the dual-light fusion algorithm; when the judgment result is consistent with the preset judgment result, generate an alarm message.

[0098] An embodiment of the present application further provides a storage medium, which includes a stored program, wherein the program executes any of the above methods when it is run.

[0099] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0100] S1, controlling the target device to send a millimeter wave to a target area, and determining a first object in the target area according to a reflected millimeter wave corresponding to the millimeter wave;

[0101] S2, performing three-dimensional reconstruction on the target area, and screening the first object according to the three-dimensionally reconstructed target area to determine a target object in the target area;

[0102] S3, controlling the dual-light gimbal on the target device to shoot the target object to obtain image information of the target object.

[0103] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0104] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0105] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0106] S1, controlling the target device to send a millimeter wave to a target area, and determining a first object in the target area according to a reflected millimeter wave corresponding to the millimeter wave;

[0107] S2, performing three-dimensional reconstruction on the target area, and screening the first object according to the three-dimensionally reconstructed target area to determine a target object in the target area;

[0108] S3, controlling the dual-light gimbal on the target device to shoot the target object to obtain image information of the target object.

[0109] An embodiment of the present application further provides a computer program product, including a computer program, which is used by a processor to execute the steps in any of the above method embodiments.

[0110] Optionally, in this embodiment, the computer program product may be executed by a processor to perform the following steps:

[0111] S1, controlling the target device to send a millimeter wave to a target area, and determining a first object in the target area according to a reflected millimeter wave corresponding to the millimeter wave;

[0112] S2, performing three-dimensional reconstruction on the target area, and screening the first object according to the three-dimensionally reconstructed target area to determine a target object in the target area;

[0113] S3, controlling the dual-light gimbal on the target device to shoot the target object to obtain image information of the target object.

[0114] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0115] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0116] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0117] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for acquiring image information, characterized in that: include: Controlling the target device to send millimeter waves to a target area, and determining a first object in the target area according to a reflected millimeter wave corresponding to the millimeter wave; Performing three-dimensional reconstruction on the target area, and screening the first object according to the three-dimensionally reconstructed target area to determine the target object in the target area; The dual-light pan / tilt platform on the target device is controlled to shoot the target object to obtain image information of the target object.

2. The method for acquiring image information according to claim 1, characterized in that: Performing three-dimensional reconstruction on the target area includes: Determining motion state information of the first object according to the reflected millimeter wave, wherein the motion state information includes at least one of the following: a distance of the first object relative to the target device, a speed of the first object, an acceleration of the target object, and an azimuth of the target object relative to the target device; The target area is three-dimensionally reconstructed according to the motion state information, wherein each first object is each three-dimensional feature point in the target area after the three-dimensional reconstruction.

3. The method for acquiring image information according to claim 2, characterized in that: Screening the first object according to the three-dimensionally reconstructed target area to determine the target object in the target area includes: Determine a curvature change value and a height of each first object according to the three-dimensional feature points of each first object, and obtain a speed of each first object; The first object is screened according to the curvature change value, the height, and the speed to determine the target object.

4. The method for acquiring image information according to claim 3, characterized in that: Screening the first object according to the curvature change value, the height, and the speed to determine the target object includes: Filtering out, from the first objects, a second object whose curvature change value is less than or equal to a preset curvature change value according to the curvature change value; Filtering out third objects whose heights are less than or equal to a preset height from the second objects according to the heights; According to the speed, a target object whose speed is less than or equal to a preset speed is screened out from the third objects.

5. The method for acquiring image information according to claim 1, characterized in that: Controlling the dual-light pan / tilt platform on the target device to shoot the target area includes: determining a first angle of the target object relative to the target device, and determining a distance of the target object relative to the target device; adjusting a first shooting direction of the dual-light gimbal according to the first angle, and adjusting a first focal length of the dual-light gimbal according to the angle and the distance when it is determined that the distance is less than or equal to a preset distance; The dual-light gimbal is controlled to shoot the target object based on the first shooting direction and the first focal length.

6. The method for acquiring image information according to claim 5, characterized in that: After determining the distance of the target object relative to the target device, the method further includes: When it is determined that the distance is greater than the preset distance, controlling the target device to move toward the target object; When it is determined that the moving distance of the target device is less than or equal to the maximum moving distance of the target device, determining the real-time distance between the target device and the target object; when it is determined that the real-time distance is less than or equal to the preset distance, determining a second angle of the target object relative to the target device; adjusting a second shooting direction of the dual-light gimbal according to the second angle, and adjusting a second focal length of the dual-light gimbal according to the second shooting direction and the real-time distance; The dual-light gimbal is controlled to shoot the target device based on the second shooting direction and the second focal length.

7. The method for acquiring image information according to claim 1, characterized in that: After acquiring the image information of the target object, the method further includes: Judging the image information according to a dual-light fusion algorithm; When the judgment result is consistent with the preset judgment result, an alarm message is generated.

8. A device for acquiring image information, characterized in that: include: A first control module, used to control the target device to send millimeter waves to a target area, and determine a first object in the target area according to a reflected millimeter wave corresponding to the millimeter wave; a screening module, configured to perform three-dimensional reconstruction of the target area, and screen the first object according to the three-dimensionally reconstructed target area to determine a target object in the target area; The second control module is used to control the dual-light pan / tilt platform on the target device to shoot the target object to obtain image information of the target object.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 7 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images