Monocular camera-based 3D fire fighting access occupation detection method and apparatus, and electronic device

Abstract

The invention discloses a monocular camera-based 3D fire fighting access occupation detection method and apparatus, and an electronic device. Comprising the steps that an image collected by a monocular camera on a fire fighting access is obtained, and the image is collected in the vehicle driving direction of the fire fighting access; determining a first depth image corresponding to the image, wherein the first depth image represents depth data of image content in the image; environment information of an image acquisition period of the image is acquired, depth adjustment is performed on the first depth image based on the environment information, a second depth image is obtained, and different environment information corresponds to different depth adjustment strategies; and performing target detection on the second depth image, determining a target detection result, and determining an occupation detection result of the fire fighting access based on the target detection result. According to the scheme, the influence of environment information on the depth image is considered, the depth image is dynamically adjusted, the occupation detection result is determined according to the adjusted depth image, and the accuracy of the detection result in a complex environment is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a 3D fire passage occupancy detection method, device and electronic equipment based on a monocular camera. Background Art

[0002] With the development of computer vision image processing technology, fire escape occupancy detection can be achieved by analyzing information in images. However, in actual fire escape detection scenarios, environmental factors often affect accurate image information extraction. For example, environmental factors such as lighting, sunlight, rain, and snow have a significant impact on object detection based on monocular cameras.

[0003] When dealing with these environmental factors, existing technologies find it difficult to comprehensively and accurately adjust the depth and entry and exit time of the target object, affecting the reliability of the detection results and leading to more false detections and false alarms. Summary of the Invention

[0004] The present invention provides a 3D fire passage occupancy detection method, device and electronic equipment based on a monocular camera to solve the problem of inaccurate fire passage occupancy detection results under the influence of environmental factors.

[0005] According to one aspect of the present invention, a 3D fire passage occupancy detection method based on a monocular camera is provided, comprising:

[0006] Acquire an image of a fire passage captured by a monocular camera, wherein the image is captured along the direction of vehicle travel in the fire passage;

[0007] Determining a first depth image corresponding to the image, wherein the first depth image represents depth data of image content in the image;

[0008] Acquiring environmental information during an image acquisition period of the image, and performing depth adjustment on the first depth image based on the environmental information to obtain a second depth image, wherein different environmental information corresponds to different depth adjustment strategies;

[0009] Perform target detection on the second depth image to determine a target detection result, and determine an occupancy detection result of the fire passage based on the target detection result.

[0010] Optionally, determining a first depth image corresponding to the image includes: calling a preset depth estimation model, performing depth estimation processing on the image using the preset depth estimation model, and obtaining the first depth image.

[0011] Optionally, the environmental information includes at least one of the following: lighting information and weather type; the lighting information is collected by the light sensing component in the monocular camera; the weather type is read from a set application based on the location information of the fire escape; or, is obtained by identifying the weather type of the image.

[0012] Optionally, depth adjustment is performed on the first depth image based on environmental information to obtain a second depth image, including: comparing the lighting information with a preset lighting intensity threshold to determine the lighting intensity type corresponding to the lighting information; determining the weight data corresponding to the lighting intensity type based on a mapping relationship between the lighting intensity type and the weight data; and obtaining first depth correction data corresponding to the fire channel, wherein the first depth correction data is determined based on scene prior information corresponding to the fire channel, and the scene prior information includes one or more of scene category information, object semantic information, occlusion relationship between objects and lighting information; and fusing the first depth image and the first depth correction data based on the weight data to obtain the second depth image.

[0013] Optionally, depth adjustment is performed on the first depth image based on environmental information to obtain a second depth image, including: when the weather type is a set weather type, obtaining depth compensation data or a depth correction coefficient corresponding to the set weather type; the set weather type includes a rainy day type or a snowy day type; correcting the first depth image based on the depth correction coefficient to obtain the second depth image; or compensating the first depth image based on the depth compensation data to obtain the second depth image.

[0014] Optionally, depth adjustment is performed on the first depth image based on environmental information to obtain a second depth image, including: performing depth adjustment on the first depth image based on a depth adjustment strategy corresponding to the lighting information to obtain a third depth image; performing depth adjustment on the first depth image based on a depth adjustment strategy corresponding to the weather type to obtain a fourth depth image; and performing fusion processing on the third depth image and the fourth depth image to obtain a second depth image.

[0015] Optionally, target detection is performed on the second depth image to determine the target detection result, and the occupancy detection result of the fire passage is determined based on the target detection result, including: performing target detection on the second depth image data through a pre-trained target detection model to obtain the target detection result; if the target detection result is that the target detection object exists, then the occupancy detection result of the fire passage is determined to be occupied; if the target detection result is that the target detection object does not exist, then the occupancy detection result of the fire passage is determined to be unoccupied.

[0016] According to another aspect of the present invention, a 3D fire passage occupancy detection device based on a monocular camera is provided, comprising:

[0017] An image acquisition module is used to acquire an image of the fire passage captured by a monocular camera, wherein the image is an image captured along the direction of vehicle travel of the fire passage;

[0018] A first depth image determining module, configured to determine a first depth image corresponding to an image, wherein the first depth image represents depth data of image content in the image;

[0019] a second depth image determination module, configured to obtain environmental information of an image acquisition period, and perform depth adjustment on the first depth image based on the environmental information to obtain a second depth image, wherein different environmental information corresponds to different depth adjustment strategies;

[0020] The occupancy detection result determination module is used to perform target detection on the second depth image, determine the target detection result, and determine the occupancy detection result of the fire passage based on the target detection result.

[0021] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0022] at least one processor; and

[0023] a memory communicatively connected to at least one processor; wherein,

[0024] The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the 3D fire channel occupancy detection method based on a monocular camera of any embodiment of the present invention.

[0025] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions, and the computer instructions are used to enable a processor to implement the 3D fire channel occupancy detection method based on a monocular camera according to any embodiment of the present invention when executed.

[0026] The technical solution of the embodiment of the present invention obtains an image of a fire passage captured by a monocular camera, wherein the image is an image captured along the direction of vehicle travel of the fire passage; by obtaining the image of the fire passage, a data basis is provided for subsequently determining the occupancy detection result of the fire passage; a first depth image corresponding to the image is determined, wherein the first depth image represents depth data of the image content in the image; by preliminarily determining the depth image of the image, the depth data of the image content in the image can be obtained, which can achieve data dimensionality reduction to a certain extent, reduce the amount of data processing, and also provide a data basis for subsequent adjustment of the depth information; environmental information of the image during the image capture period is obtained, and the depth of the first depth image is adjusted based on the environmental information to obtain a second depth image, wherein different environmental information corresponds to different depth adjustment strategies; by considering the impact of the environmental information on the depth image, the depth image is dynamically adjusted according to the depth adjustment strategy corresponding to the environmental information to obtain more accurate depth data, thereby improving the accuracy of the depth image and helping to improve the accuracy of the detection results in complex environments; target detection is performed on the second depth image to determine the target detection result, the fire passage occupancy detection result is determined based on the target detection result, and the channel occupancy detection result is determined by performing target detection on the adjusted depth image, thereby helping to improve the accuracy of the detection result. This solution optimizes the depth image of the image captured by the monocular camera based on environmental information, realizes dynamic adjustment of the depth image, improves the accuracy of the depth image, and determines the occupancy detection result based on the adjusted depth image. It solves the problem of inaccurate fire passage occupancy detection results under the influence of environmental factors, significantly improves the accuracy of detection results in complex environments, enhances the adaptability of the detection system to different environments, and reduces misjudgments and missed detections caused by environmental interference.

[0027] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 This is a flow chart of a 3D fire passage occupancy detection method based on a monocular camera provided in Example 1 of the present invention;

[0030] Figure 2This is a flow chart of a 3D fire passage occupancy detection method based on a monocular camera provided in Example 2 of the present invention;

[0031] Figure 3 This is a flow chart of a 3D fire passage occupancy detection method based on a monocular camera provided in Example 3 of the present invention;

[0032] Figure 4 This is a flow chart of a 3D fire passage occupancy detection method based on a monocular camera provided in Example 4 of the present invention;

[0033] Figure 5 This is a structural diagram of a 3D fire passage occupancy detection device based on a monocular camera provided in Example 5 of the present invention;

[0034] Figure 6 The present invention is a schematic diagram of the structure of an electronic device that implements the 3D fire passage occupancy detection method based on a monocular camera according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0036] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] Example 1

[0038] Figure 1This is a flow chart of a 3D fire passage occupancy detection method based on a monocular camera provided in the first embodiment of the present invention. This embodiment is applicable to the situation of occupancy detection of fire passages. The method can be performed by a 3D fire passage occupancy detection device based on a monocular camera. The 3D fire passage occupancy detection device based on a monocular camera can be implemented in the form of hardware and / or software. The 3D fire passage occupancy detection device based on a monocular camera can be configured in electronic devices such as computers and servers. Figure 1 As shown, the method includes:

[0039] S110. Acquire an image of the fire passage captured by a monocular camera, wherein the image is captured along a vehicle travel direction of the fire passage.

[0040] In this embodiment, monocular cameras are installed at appropriate locations on both sides of the fire lane at precisely measured heights to ensure they can clearly and comprehensively capture images of vehicles traveling along the fire lane. Specifically, the images refer to two-dimensional images captured by the monocular cameras along the direction of vehicle travel along the fire lane.

[0041] Specifically, a monocular camera installed in a fire passage captures high-resolution images along the direction of vehicle travel in the fire passage. The monocular camera can directly capture images of the fire passage to obtain an image corresponding to the fire passage at the current moment. Alternatively, the monocular camera can capture a video stream of the fire passage captured by the monocular camera and read the corresponding image from the video stream using a preset image reading algorithm. The video stream and images of the fire passage captured by the monocular camera are both captured by the monocular camera along the direction of vehicle travel in the fire passage.

[0042] In this embodiment, images are collected along the direction of vehicle travel in the fire lane. Through the collected images, the driving conditions, parking status, and whether there is illegal occupation of vehicles in the fire lane can be observed more intuitively, providing strong evidence support for subsequent management and law enforcement.

[0043] S120: Determine a first depth image corresponding to the image, where the first depth image represents depth data of image content in the image.

[0044] Specifically, a depth image can be understood as an image that contains information about the distance of objects in the scene from the camera. The depth image can be obtained by processing the image using a trained depth data estimation model. Alternatively, a multi-frame image-based method can be used to calculate depth using optical flow or stereo vision, utilizing the motion information of multiple frames captured at different times by a monocular camera. This can improve accuracy and robustness. The specific depth image determination algorithm is determined based on the actual scene and is not limited here. The first depth map image is a preliminary representation of the depth information of the fire escape scene, specifically representing the depth data of the image content.

[0045] Specifically, when an image of a fire escape is captured by a monocular camera, a depth data determination algorithm can be used to process the image to obtain a corresponding first depth image, which can be used to simulate a 3D effect of the fire escape.

[0046] In this embodiment, by preliminarily determining the depth image of the image, the depth data of the image content in the image can be obtained, which can achieve data dimensionality reduction to a certain extent, reduce the amount of data processing, and also provide a data basis for subsequent adjustment of the depth information.

[0047] Optionally, determining a first depth image corresponding to the image includes: calling a preset depth estimation model, performing depth estimation processing on the image using the preset depth estimation model, and obtaining the first depth image.

[0048] Among them, the preset depth estimation model specifically refers to a model for predicting the depth information of objects in an image or scene, and is pre-trained. It can be directly called when performing depth estimation processing. For example, a depth estimation model can be constructed based on a convolutional network. The constructed depth estimation model is trained on a large number of monocular and corresponding depth images to learn the mapping relationship, thereby obtaining a trained depth estimation model.

[0049] Specifically, a preset depth estimation model is called, and an image is input into the preset depth estimation model, so that the preset depth estimation model infers the distance of each pixel in the scene relative to the camera based on the input image, thereby generating a corresponding depth image, that is, obtaining a first depth image.

[0050] S130 , obtaining environmental information of an image acquisition period, and performing depth adjustment on the first depth image based on the environmental information to obtain a second depth image, wherein different environmental information corresponds to different depth adjustment strategies.

[0051] Environmental information can be specifically understood as information that affects image quality. Optionally, the environmental information includes at least one of the following: lighting information and weather type; lighting information is collected by the light-sensing component of the monocular camera; the weather type is read from a setting application based on the location information of the fire escape; or it is obtained by identifying the weather type of the image. The setting application specifically refers to program code that provides timely and accurate weather information. The setting application can be called using the location information of the fire escape as an input parameter to provide feedback on the weather type for the corresponding location. Alternatively, a pre-trained weather type determination model can be called to identify the weather type of the image using the weather type determination model to obtain the corresponding weather type. Different depth adjustment strategies are set for different environmental information. The second depth image specifically represents the image adjusted according to the corresponding depth adjustment strategy. The depth adjustment strategy specifically refers to a set of methods used to optimize and improve processing processes related to depth information. Depth adjustment strategies can be pre-set for different environmental information, and then the corresponding depth adjustment strategies can be matched based on the environmental information, facilitating flexible depth adjustment based on different environmental information.

[0052] Specifically, the location information of the fire passage and the image acquisition period of the image can be analyzed and processed by setting an application, and the environmental information of the image acquisition period can be output. The environmental information includes but is not limited to lighting information and weather type. According to the environmental information, a match is performed in the preset depth adjustment strategy library to obtain the depth adjustment strategy corresponding to the environmental information, and then the depth of the first depth image is adjusted according to the depth adjustment strategy to obtain the second depth image.

[0053] In this embodiment, by considering the impact of environmental information on the depth image, the depth image is dynamically adjusted according to the depth adjustment strategy corresponding to the environmental information to obtain more accurate depth data, thereby improving the accuracy of the depth image, helping to improve the accuracy of detection results in complex environments, enhancing the adaptability of the detection system to different environments, and reducing misjudgments and missed judgments caused by environmental interference.

[0054] S140: Perform target detection on the second depth image to determine a target detection result, and determine an occupancy detection result of the fire passage based on the target detection result.

[0055] Among them, target detection can be specifically understood as a process aimed at identifying specific target objects in images or videos and determining their positions and categories. The depth image can be processed by a target detection algorithm to obtain the corresponding target detection results. Exemplarily, the depth image can be processed for target detection by a target detection model to obtain the corresponding target detection results. The target detection model includes but is not limited to YOLOv7 (You Only Look Once v7), YOLOv8 (YouOnly Look Once v10) and SSD (Single Shot MultiBox Detector) models. The target detection result specifically represents the structured information output by the target detection model after processing the input image or video, mainly including the detected object position and category. These results are used to describe the existence and attributes of each object in the image.

[0056] Specifically, target detection processing is performed on the second depth image through a preset target detection algorithm to identify whether there is a target object in the depth image. If there is a target object, the location information and category information of the object are identified, and then the occupancy detection result of the fire passage is determined based on the identified target detection result. For example, it can be set that if any target object is detected, the occupancy detection result of the fire passage can be determined as the fire passage is occupied. If no target object is detected, the occupancy detection result of the fire passage can be determined as the fire passage is not occupied.

[0057] Optionally, target detection is performed on the second depth image to determine the target detection result, and the occupancy detection result of the fire passage is determined based on the target detection result, including: performing target detection on the second depth image data through a pre-trained target detection model to obtain the target detection result; if the target detection result is that the target detection object exists, then the occupancy detection result of the fire passage is determined to be occupied; if the target detection result is that the target detection object does not exist, then the occupancy detection result of the fire passage is determined to be unoccupied.

[0058] Specifically, a pre-trained target detection model is called, the second depth image is input into the target detection model, the second depth image is processed by the target detection model, and the target detection model outputs the target detection result. According to the output result of the target detection model, it is checked whether there is a target detection object. The target detection object includes but is not limited to vehicles and obstacles in the fire passage. When the target detection result is that there is a target detection object, it means that there are target objects in the scene corresponding to the second depth image. Since these target objects may affect the normal use of the fire passage, the occupancy detection result of the fire passage can be determined to be occupied at this time. When the target detection result is that there is no target detection object, it means that no target object that may occupy the fire passage is detected in the scene, and the occupancy detection result of the fire passage can be determined to be unoccupied.

[0059] The technical solution of this embodiment is to obtain an image of the fire escape captured by a monocular camera; determine a first depth image corresponding to the image; obtain environmental information of the image during the image capture period, and perform depth adjustment on the first depth image based on the environmental information to obtain a second depth image, wherein different environmental information corresponds to different depth adjustment strategies; perform target detection on the second depth image, determine the target detection result, and determine the fire escape occupancy detection result based on the target detection result. This solution optimizes the depth image of the image captured by the monocular camera based on environmental information, realizes dynamic adjustment of the depth image, and determines the occupancy detection result based on the adjusted depth image. This solves the problem of inaccurate fire escape occupancy detection results under the influence of environmental factors, significantly improves the accuracy of detection results in complex environments, enhances the adaptability of the detection system to different environments, and reduces misjudgments and missed judgments caused by environmental interference.

[0060] Example 2

[0061] Figure 2 This is a flow chart of a 3D fire passage occupancy detection method based on a monocular camera provided by the second embodiment of the present invention. This embodiment is a further optimization of the method of the above embodiment. Optionally, the environmental information includes lighting information; the lighting information is compared with a preset lighting intensity threshold to determine the lighting intensity type corresponding to the lighting information; based on the mapping relationship between the lighting intensity type and the weight data, the weight data corresponding to the lighting intensity type is determined; and the first depth correction data corresponding to the fire passage is obtained; the first depth image and the first depth correction data are fused based on the weight data to obtain a second depth image. Figure 2 As shown, the method includes:

[0062] S210. Acquire an image of the fire passage captured by a monocular camera, wherein the image is captured along the direction of vehicle travel of the fire passage.

[0063] S220: Determine a first depth image corresponding to the image, where the first depth image represents depth data of image content in the image.

[0064] S230 , obtaining illumination information of the image during an image acquisition period, comparing the illumination information with a preset illumination intensity threshold, and determining an illumination intensity type corresponding to the illumination information.

[0065] The illumination information specifically represents the illumination intensity data of the scene corresponding to the second depth image. The illumination information is collected by a light-sensing component in a monocular camera. Exemplarily, the light-sensing component may be an ambient light sensor. Changes in illumination intensity, from strong direct light to dim light, alter the brightness and contrast of the image, affecting the feature extraction of the target object and thereby interfering with the calculation of depth information. The illumination intensity can be divided into strong light type and weak light type according to the degree of influence of the illumination intensity. Different illumination intensity types correspond to different illumination intensity thresholds. Preset illumination intensity thresholds can be set for different illumination intensity types to determine the illumination intensity type corresponding to the illumination information.

[0066] Specifically, the lighting information is collected by the light sensing component in the monocular camera, the preset light intensity threshold corresponding to the current scene is obtained from the preset storage space, the lighting information is compared with the preset light intensity threshold, and the lighting intensity type corresponding to the preset light intensity threshold that is successfully matched is determined as the light intensity type corresponding to the lighting information.

[0067] S240. Determine the weight data corresponding to the light intensity type based on the mapping relationship between the light intensity type and the weight data; and obtain first depth correction data corresponding to the fire channel.

[0068] The first depth correction data is determined based on scene prior information corresponding to the fire escape. This scene prior information includes one or more of scene category information, object semantic information, inter-object occlusion relationships, and lighting information. Scene category information can clarify the type of environment the fire escape is located in, such as whether it is in a different architectural setting, such as an office building or a residential building. Object semantic information can identify various objects that may exist in and around the fire escape, such as firefighting equipment, signs, debris, and vehicles. Inter-object occlusion relationships help identify objects that may interfere with or obstruct the depth information of the fire escape, such as goods stacked at the entrance of the escape that may block part of the escape. Lighting information can reflect the lighting conditions in the scene, such as whether there is sufficient light, dim light, or partial shadows. By comprehensively considering one or more of these scene prior information, the depth data related to the fire escape is corrected to improve the accuracy and reliability of the depth data's description of the fire escape scene, providing more valuable basic information for subsequent analysis and judgment based on depth data. A model for determining depth correction data based on scene prior information is established using machine learning or deep learning algorithms. The collected scene prior information and the corresponding depth image data are used as training data, and the training model learns to determine the depth correction data based on the prior information. For example, a depth correction data determination model can be constructed based on a convolutional neural network (CNN) to predict accurate depth correction data.

[0069] Specifically, a mapping relationship between the light intensity type and the weight data is obtained from a preset storage space, and the light intensity type is matched with the mapping relationship to obtain weight data corresponding to the light intensity type; and scene prior information corresponding to the current fire escape scene is obtained, and a depth correction data determination model is called to process the scene prior information to obtain first depth correction data;

[0070] S250 : Fusing the first depth image and the first depth correction data based on the weight data to obtain a second depth image.

[0071] Specifically, for each pixel position in the image, the first depth image and the first depth correction data are weighted and summed using the weight data to obtain the second depth image. adj1 The calculation formula is as follows:

[0072] D adj =αD img +(1-α)D pri ;

[0073] Among them, D img is the first depth image, D priis first depth correction data calculated based on scene prior information, wherein the first correction data is a depth image with the same size as the first depth image, and α is weight data.

[0074] S260: Perform target detection on the second depth image to determine a target detection result, and determine an occupancy detection result of the fire passage based on the target detection result.

[0075] The technical solution of this embodiment is to obtain an image of the fire escape captured by a monocular camera; determine the first depth image corresponding to the image; compare the illumination information with a preset illumination intensity threshold to determine the illumination intensity type corresponding to the illumination information; determine the weight data corresponding to the illumination intensity type based on the mapping relationship between the illumination intensity type and the weight data; and obtain the first depth correction data corresponding to the fire escape; fuse the first depth image and the first depth correction data based on the weight data to obtain a second depth image; perform target detection on the second depth image to determine the target detection result, and determine the occupancy detection result of the fire escape based on the target detection result. This solution optimizes the depth image of the image captured by the monocular camera based on environmental information, realizes dynamic adjustment of the depth image, and determines the occupancy detection result based on the adjusted depth image, solving the problem of inaccurate fire escape occupancy detection results under the influence of environmental factors, significantly improving the accuracy of detection results in complex environments, enhancing the adaptability of the detection system to different environments, and reducing misjudgments and missed judgments caused by environmental interference.

[0076] Example 3

[0077] Figure 3 This is a flowchart of a 3D fire passage occupancy detection method based on a monocular camera provided by the third embodiment of the present invention. This embodiment is a further optimization of the method of the above embodiment. Optionally, the environmental information includes the weather type; when the weather type is the set weather type, the depth compensation data or depth correction coefficient corresponding to the set weather type is obtained; the set weather type includes a rainy day type or a snowy day type; the first depth image is corrected based on the depth correction coefficient to obtain a second depth image; or the first depth image is compensated based on the depth compensation data to obtain a second depth image. Figure 3 As shown, the method includes:

[0078] S310. Acquire an image of the fire passage captured by a monocular camera, wherein the image is captured along the direction of vehicle travel in the fire passage.

[0079] S320: Determine a first depth image corresponding to the image, where the first depth image represents depth data of image content in the image.

[0080] S330, obtaining the weather type of the image acquisition period, and when the weather type is a set weather type, obtaining depth compensation data or a depth correction coefficient corresponding to the set weather type; the set weather type includes a rainy day type or a snowy day type.

[0081] It should be noted that for rainy days, the depth value of the target object may deviate due to occlusion by raindrops and light refraction. Based on the raindrop density estimate, the raindrop density can be calculated by the ratio of raindrop pixels in the image. Assuming the raindrop density is r, the correction coefficient k(r) corresponding to the rainy day type can be obtained by fitting historical data. For example, the correction coefficient can be expressed as k(r) = 1 + 0.5r. For snowy days, the accumulation and scattering of snowflakes complicates the depth calculation of the target object. For targets partially obscured by snow, the characteristics of the unobstructed portion of the target in multiple frames of image are used, combined with scene prior information, to estimate the depth value of the obscured portion to obtain depth compensation data.

[0082] Specifically, when the weather type is rainy, the raindrop density is determined based on the first depth image, and then the depth correction coefficient corresponding to the first depth image is calculated according to the correction coefficient calculation formula. When the weather type is snowy, multiple frames adjacent to the first depth image can be obtained, and the depth image corresponding to each frame can be determined. The depth compensation data calculation algorithm is called to process the first depth image and the multiple adjacent depth images, as well as the scene prior information, to obtain depth compensation data corresponding to the first depth image.

[0083] S340 : Correct the first depth image based on the depth correction coefficient to obtain a second depth image; or compensate the first depth image based on the depth compensation data to obtain a second depth image.

[0084] Specifically, when depth correction data corresponding to the first depth image is obtained, a product operation is performed on the depth correction coefficient and the first depth image to obtain the second depth image. Alternatively, when depth compensation data corresponding to the first depth image is obtained, a sum operation is performed on the first depth image and the corresponding depth compensation data to obtain the second depth image.

[0085] S350: Perform target detection on the second depth image to determine a target detection result, and determine an occupancy detection result of the fire passage based on the target detection result.

[0086] The technical solution of this embodiment is to obtain an image of the fire escape captured by a monocular camera; determine a first depth image corresponding to the image; when the weather type is a set weather type, obtain depth compensation data or a depth correction coefficient corresponding to the set weather type; set the weather type to include a rainy day type or a snowy day type; correct the first depth image based on the depth correction coefficient to obtain a second depth image; or compensate the first depth image based on the depth compensation data to obtain a second depth image; perform target detection on the second depth image to determine the target detection result, and determine the fire escape occupancy detection result based on the target detection result. This solution optimizes the depth image of the image captured by the monocular camera based on environmental information, realizes dynamic adjustment of the depth image, and determines the occupancy detection result based on the adjusted depth image. This solves the problem of inaccurate fire escape occupancy detection results under the influence of environmental factors, significantly improves the accuracy of detection results in complex environments, enhances the adaptability of the detection system to different environments, and reduces misjudgments and missed judgments caused by environmental interference.

[0087] Example 4

[0088] Figure 4 This is a flowchart of a 3D fire passage occupancy detection method based on a monocular camera provided by the fourth embodiment of the present invention. This embodiment is a further optimization of the method of the above embodiment. Optionally, the environmental information includes lighting information and weather type; the first depth image is depth-adjusted based on the depth adjustment strategy corresponding to the lighting information to obtain a third depth image; the first depth image is depth-adjusted based on the depth adjustment strategy corresponding to the weather type to obtain a fourth depth image; the third depth image and the fourth depth image are fused to obtain a second depth image. Figure 4 As shown, the method includes:

[0089] S410. Acquire an image of the fire passage captured by a monocular camera, wherein the image is captured along a vehicle travel direction of the fire passage.

[0090] S420: Determine a first depth image corresponding to the image, where the first depth image represents depth data of image content in the image.

[0091] S430: Perform depth adjustment on the first depth image based on a depth adjustment strategy corresponding to the illumination information to obtain a third depth image.

[0092] Specifically, when the acquired environmental information is illumination information, a depth adjustment strategy corresponding to the illumination information is invoked, and the depth of the first depth image is adjusted according to the depth adjustment strategy to obtain a third depth image. This can effectively improve the quality of the depth image, overcome the influence of illumination on the quality of the depth image, and enhance the accuracy of the depth image, providing a more reliable data foundation for subsequent target detection.

[0093] S440: Perform depth adjustment on the first depth image based on a depth adjustment strategy corresponding to the weather type to obtain a fourth depth image.

[0094] Specifically, when the acquired environmental information is a weather type, the depth adjustment strategy corresponding to the weather type is invoked, and the depth of the first depth image is adjusted according to the depth adjustment strategy to obtain a fourth depth image. This can effectively improve the quality of the depth image, overcome the impact of weather factors on the quality of the depth image, and improve the accuracy of the depth image, providing a more reliable data foundation for subsequent target detection.

[0095] S450: Perform fusion processing based on the third depth image and the fourth depth image to obtain a second depth image.

[0096] Based on the above embodiment, taking into account the impact of lighting factors and weather factors on the quality of the depth image, the third depth image and the fourth depth image can be fused to complete the adjustment of the first depth image to obtain the second depth image. For example, the depth values of the corresponding pixels of the two depth images can be weighted and summed according to certain weights to obtain the depth values of the corresponding pixels in the fused image. Alternatively, the two depth images can be first constructed into pyramid structures, which include but are not limited to Gaussian pyramids and Laplacian pyramids. Then, a fusion operation is performed on each layer of the pyramid. A weighted average fusion rule can be used. Finally, the fused image is reconstructed through the pyramid to obtain a fused depth image, i.e., the second depth image.

[0097] In this embodiment, by fusing the third depth image and the fourth depth image to obtain the second depth image, the complementary information between the depth images can be integrated to improve the quality of the depth image, and the influence of lighting and weather factors on the quality of the depth image can be overcome at the same time, thereby improving the accuracy of the depth image and providing better and more comprehensive data support for subsequent analysis and processing.

[0098] S460: Perform target detection on the second depth image to determine a target detection result, and determine an occupancy detection result of the fire passage based on the target detection result.

[0099] The technical solution of this embodiment is to obtain an image of the fire escape captured by a monocular camera; determine a first depth image corresponding to the image; perform depth adjustment on the first depth image based on a depth adjustment strategy corresponding to the illumination information to obtain a third depth image; perform depth adjustment on the first depth image based on a depth adjustment strategy corresponding to the weather type to obtain a fourth depth image; perform fusion processing based on the third depth image and the fourth depth image to obtain a second depth image; perform target detection on the second depth image to determine the target detection result, and determine the occupancy detection result of the fire escape based on the target detection result. This solution optimizes the depth image of the image captured by the monocular camera based on environmental information, realizes dynamic adjustment of the depth image, and determines the occupancy detection result based on the adjusted depth image, solving the problem of inaccurate fire escape occupancy detection results under the influence of environmental factors, significantly improving the accuracy of detection results in complex environments, enhancing the adaptability of the detection system to different environments, and reducing misjudgments and missed judgments caused by environmental interference.

[0100] Example 5

[0101] Figure 5 This is a structural diagram of a 3D fire passage occupancy detection device based on a monocular camera provided in the fifth embodiment of the present invention. Figure 5 As shown, the device includes:

[0102] An image acquisition module 510 is configured to acquire an image of the fire passage captured by a monocular camera, wherein the image is captured along the direction of vehicle travel of the fire passage;

[0103] A first depth image determining module 520 is configured to determine a first depth image corresponding to the image, wherein the first depth image represents depth data of image content in the image;

[0104] A second depth image determination module 530 is configured to obtain environmental information during an image acquisition period, and perform depth adjustment on the first depth image based on the environmental information to obtain a second depth image, wherein different environmental information corresponds to different depth adjustment strategies;

[0105] The occupancy detection result determination module 540 is configured to perform target detection on the second depth image, determine a target detection result, and determine an occupancy detection result of the fire passage based on the target detection result.

[0106] The technical solution of this embodiment is to obtain an image of the fire passage captured by a monocular camera through an image acquisition module, wherein the image is an image captured along the direction of vehicle travel of the fire passage; a first depth image determination module determines a first depth image corresponding to the image, wherein the first depth image represents the depth data of the image content in the image; a second depth image determination module obtains environmental information of the image during the image acquisition period, and performs depth adjustment on the first depth image based on the environmental information to obtain a second depth image, wherein different environmental information corresponds to different depth adjustment strategies; an occupancy detection result determination module performs target detection on the second depth image, determines the target detection result, and determines the occupancy detection result of the fire passage based on the target detection result. This solution optimizes the depth image of the image captured by the monocular camera based on environmental information, realizes dynamic adjustment of the depth image, and determines the occupancy detection result based on the adjusted depth image. This solves the problem of inaccurate fire passage occupancy detection results under the influence of environmental factors, significantly improves the accuracy of detection results in complex environments, enhances the adaptability of the detection system to different environments, and reduces misjudgments and missed judgments caused by environmental interference.

[0107] Based on the above embodiment, optionally, the first depth image determining module 520 is specifically configured to call a preset depth estimation model, perform depth estimation processing on the image through the preset depth estimation model, and obtain a first depth image.

[0108] Optionally, the environmental information includes at least one of the following: lighting information and weather type; the lighting information is collected by the light-sensing component in the monocular camera; the weather type is read from a set application based on the location information of the fire escape; or obtained by identifying the weather type of the image. The second depth image determination module 530 is specifically used to compare the lighting information with a preset lighting intensity threshold to determine the lighting intensity type corresponding to the lighting information; based on the mapping relationship between the lighting intensity type and the weight data, determine the weight data corresponding to the lighting intensity type; and obtain the first depth correction data corresponding to the fire escape, wherein the first depth correction data is determined based on the scene prior information corresponding to the fire escape, and the scene prior information includes one or more of scene category information, object semantic information, occlusion relationship between objects and lighting information; based on the weight data, the first depth image and the first depth correction data are fused to obtain the second depth image.

[0109] Optionally, the second depth image determination module 530 is further specifically used to obtain depth compensation data or depth correction coefficient corresponding to the set weather type when the weather type is the set weather type; the set weather type includes a rainy day type or a snowy day type; based on the depth correction coefficient, the first depth image is corrected to obtain a second depth image; or, based on the depth compensation data, the first depth image is compensated to obtain a second depth image.

[0110] Optionally, the second depth image determination module 530 is further specifically used to perform depth adjustment on the first depth image based on the depth adjustment strategy corresponding to the lighting information to obtain a third depth image; perform depth adjustment on the first depth image based on the depth adjustment strategy corresponding to the weather type to obtain a fourth depth image; and perform fusion processing based on the third depth image and the fourth depth image to obtain a second depth image.

[0111] Optionally, the occupancy detection result determination module 540 is specifically used to perform target detection on the second depth image data through a pre-trained target detection model to obtain a target detection result; if the target detection result is that there is a target detection object, the occupancy detection result of the fire passage is determined to be occupied; if the target detection result is that there is no target detection object, the occupancy detection result of the fire passage is determined to be unoccupied.

[0112] The 3D fire passage occupancy detection device based on a monocular camera provided in an embodiment of the present invention can execute the 3D fire passage occupancy detection method based on a monocular camera provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0113] Example 6

[0114] Figure 6 1 is a structural diagram of an electronic device provided in Example 6 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0115] like Figure 6As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0116] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0117] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the monocular camera-based 3D fire escape occupancy detection method.

[0118] In some embodiments, the 3D fire lane occupancy detection method based on a monocular camera can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the 3D fire lane occupancy detection method based on a monocular camera described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the 3D fire lane occupancy detection method based on a monocular camera by any other appropriate means (for example, by means of firmware).

[0119] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0120] The computer programs for implementing the monocular camera-based 3D fire escape occupancy detection method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a standalone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0121] Example 7

[0122] Embodiment 7 of the present invention further provides a computer-readable storage medium storing computer instructions, which are used to cause a processor to execute a 3D fire escape occupancy detection method based on a monocular camera, the method comprising:

[0123] Acquire an image of a fire passage captured by a monocular camera, wherein the image is captured along the direction of vehicle travel in the fire passage;

[0124] Determining a first depth image corresponding to the image, wherein the first depth image represents depth data of image content in the image;

[0125] Acquiring environmental information during an image acquisition period of the image, and performing depth adjustment on the first depth image based on the environmental information to obtain a second depth image, wherein different environmental information corresponds to different depth adjustment strategies;

[0126] Perform target detection on the second depth image to determine a target detection result, and determine an occupancy detection result of the fire passage based on the target detection result.

[0127] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0128] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0129] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0130] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0131] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0132] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A 3D fire passage occupancy detection method based on a monocular camera, characterized in that: include: Acquire an image of a fire passage captured by a monocular camera, wherein the image is captured along a vehicle travel direction of the fire passage; Determining a first depth image corresponding to the image, wherein the first depth image represents depth data of image content in the image; Acquiring environmental information during an image acquisition period of the image, and performing depth adjustment on the first depth image based on the environmental information to obtain a second depth image, wherein different environmental information corresponds to different depth adjustment strategies; Perform target detection on the second depth image to determine a target detection result, and determine an occupancy detection result of the fire passage based on the target detection result.

2. The method according to claim 1, characterized in that The determining a first depth image corresponding to the image includes: A preset depth estimation model is called, and depth estimation processing is performed on the image using the preset depth estimation model to obtain the first depth image.

3. The method according to claim 1, characterized in that The environmental information includes at least one of the following: lighting information and weather type; The illumination information is collected by the light sensing component in the monocular camera; The weather type is read from a setting application according to the location information of the fire escape; or obtained by performing weather type recognition on the image.

4. The method according to claim 3, characterized in that The performing depth adjustment on the first depth image based on the environmental information to obtain a second depth image includes: Comparing the illumination information with a preset illumination intensity threshold to determine the illumination intensity type corresponding to the illumination information; Determining weight data corresponding to the light intensity type based on a mapping relationship between the light intensity type and the weight data; and obtaining first depth correction data corresponding to the fire escape, wherein the first depth correction data is determined based on scene prior information corresponding to the fire escape, the scene prior information including one or more of scene category information, object semantic information, occlusion relationships between objects, and lighting information; The first depth image and the first depth correction data are fused based on the weight data to obtain the second depth image.

5. The method according to claim 3, characterized in that The performing depth adjustment on the first depth image based on the environmental information to obtain a second depth image includes: When the weather type is a set weather type, obtaining depth compensation data or a depth correction coefficient corresponding to the set weather type; the set weather type includes a rainy day type or a snowy day type; The first depth image is corrected based on the depth correction coefficient to obtain the second depth image; or the first depth image is compensated based on the depth compensation data to obtain the second depth image.

6. The method according to claim 3, characterized in that The performing depth adjustment on the first depth image based on the environmental information to obtain a second depth image includes: Performing depth adjustment on the first depth image based on a depth adjustment strategy corresponding to the illumination information to obtain a third depth image; Performing depth adjustment on the first depth image based on the depth adjustment strategy corresponding to the weather type to obtain a fourth depth image; The second depth image is obtained by performing a fusion process based on the third depth image and the fourth depth image.

7. The method according to claim 1, characterized in that The performing target detection on the second depth image to determine a target detection result, and determining an occupancy detection result of the fire passage based on the target detection result, includes: Performing target detection on the second depth image data using a pre-trained target detection model to obtain a target detection result; If the target detection result is that the target detection object exists, the occupancy detection result of the fire passage is determined to be occupied. If the target detection result is that the target detection object does not exist, the occupancy detection result of the fire passage is determined to be unoccupied.

8. A 3D fire passage occupancy detection device based on a monocular camera, characterized in that: include: An image acquisition module is used to acquire an image of the fire passage captured by a monocular camera, wherein the image is an image captured along the direction of vehicle travel of the fire passage; A first depth image determining module, configured to determine a first depth image corresponding to the image, wherein the first depth image represents depth data of image content in the image; a second depth image determination module, configured to obtain environmental information of an image acquisition period of the image, and perform depth adjustment on the first depth image based on the environmental information to obtain a second depth image, wherein different environmental information corresponds to different depth adjustment strategies; An occupancy detection result determination module is used to perform target detection on the second depth image, determine a target detection result, and determine an occupancy detection result of the fire passage based on the target detection result.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the 3D fire passage occupancy detection method based on a monocular camera according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the 3D fire passage occupancy detection method based on a monocular camera according to any one of claims 1 to 7 when executed.

Images