Security detection method and intelligent device

Abstract

The invention is suitable for the technical field of privacy security detection, and provides a security detection method and intelligent equipment, and the method comprises the steps: obtaining one or more first images, and the first images comprise environment images outside driving equipment collected by at least one camera disposed on the driving equipment; performing feature analysis on each first image to obtain behavior features of pedestrians included in each first image; determining whether the driving equipment has a safety risk according to the behavior characteristics of the pedestrian corresponding to the plurality of first images; and under the condition that the safety risk exists, displaying an environment picture collected by at least one camera on a display screen of the driving equipment. According to the method, the safety detection precision of the driving equipment can be improved.

Description

Technical Field

[0001] This application belongs to the technical field of security detection, and particularly relates to a security detection method and an intelligent device. Background Art

[0002] With the development of intelligent vehicles, as an important means of transportation, vehicles not only undertake the function of transporting people, but are also often used as mobile offices. In this scenario, temporary business negotiations, meetings, or discussions of sensitive information may be carried out inside the vehicle. Therefore, higher requirements are put forward for the privacy security detection inside the vehicle.

[0003] Therefore, how to detect whether there are external events outside the vehicle that affect the privacy security inside the vehicle is a technical problem that needs to be solved urgently. Summary of the Invention

[0004] The embodiments of this application provide a security detection method and an intelligent device, which can improve the security detection accuracy of the driving device.

[0005] In a first aspect, the embodiments of this application provide a security detection method, including:

[0006] Obtain one or more first images, where the first images include environmental pictures outside the driving device collected by at least one camera deployed on the driving device;

[0007] Perform feature analysis on each of the first images to obtain the behavior features of pedestrians included in each of the first images;

[0008] Determine whether there is a security risk for the driving device according to the behavior features of the pedestrians corresponding to the multiple first images;

[0009] In the case of the existence of the security risk, display at least one environmental picture collected by the camera on the display screen of the driving device.

[0010] In the embodiments of this application, first, one or more images need to be obtained. These images are environmental pictures outside the driving device. Analyze the obtained images and extract the features therein. Based on these features, evaluate whether there is a security risk situation for the driving device, and determine whether there are abnormal situations such as potential peeping behaviors or other security threats. By obtaining and analyzing the environmental pictures outside the driving device in real time, and evaluating whether there is a security risk for the driving device, it is possible to effectively prevent the peeping behavior of external personnel and improve the security detection accuracy of the privacy of the people inside the vehicle.

[0011] In a second aspect, the embodiments of this application provide a security detection device, including:

[0012] An image acquisition module, configured to acquire one or more first images, where the first images include environmental images outside the driving device acquired by at least one camera deployed on the driving device;

[0013] A feature analysis module, configured to perform feature analysis on each of the first images to obtain the behavior features of pedestrians included in each of the first images;

[0014] A risk determination module, configured to determine whether there is a safety risk for the driving device according to the behavior features of the pedestrians corresponding to the multiple first images;

[0015] A risk prompt module, configured to, when there is the safety risk, display at least one environmental image acquired by the at least one camera on the display screen of the driving device.

[0016] In a third aspect, an embodiment of the present application provides an intelligent device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the safety detection method described in the first aspect or various possible implementation manners of the first aspect is implemented.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the safety detection method described in any item of the first aspect is implemented.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on an intelligent device, the intelligent device is enabled to execute the safety detection method described in the first aspect or various possible implementation manners of the first aspect.

[0019] It can be understood that the beneficial effects of the second aspect to the fifth aspect can refer to the relevant descriptions in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0021] Figure 1 It is a flowchart of the safety detection method provided by the embodiment of the present application;

[0022] Figure 2It is a schematic flowchart of obtaining behavior characteristics provided by an embodiment of the present application;

[0023] Figure 3 It is a schematic flowchart of feature analysis provided by an embodiment of the present application;

[0024] Figure 4 It is a schematic flowchart of determining security risks provided by an embodiment of the present application Figure 1 ;

[0025] Figure 5 It is a schematic flowchart of determining security risks provided by an embodiment of the present application Figure 2 ;

[0026] Figure 6 It is a schematic flowchart of performing alarm prompts provided by an embodiment of the present application;

[0027] Figure 7 It is a general schematic block diagram of security detection provided by an embodiment of the present application;

[0028] Figure 8 It is a schematic diagram of the existence of security risks provided by an embodiment of the present application;

[0029] Figure 9 It is a schematic structural diagram of a security detection device provided by an embodiment of the present application;

[0030] Figure 10 It is a schematic structural diagram of an intelligent device provided by an embodiment of the present application. Detailed implementation manners

[0031] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are proposed to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0032] It should be understood that when used in the specification and appended claims of the present application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0033] It should also be understood that the term "and / or" used in the specification and appended claims of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0034] As used in the specification and claims of this application, the term "if" may be construed, depending on the context, as "when", "once", "in response to determining", or "in response to detecting". Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be construed, depending on the context, to mean "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]".

[0035] In addition, in the description of the specification and claims of this application, the terms "first", "second", "third", etc. are used only for distinguishing descriptions and cannot be construed as indicating or implying relative importance.

[0036] References to "one embodiment" or "some embodiments" or the like described in the specification of this application mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0037] In current business activities, as an important means of transportation, vehicles not only undertake the function of transporting people, but are also often used as mobile offices. In this scenario, temporary business negotiations, meetings, or discussions of sensitive information may be carried out in the vehicle. Therefore, higher requirements are put forward for the privacy and security detection in business vehicles.

[0038] In related technologies, the camera system simply captures and displays images without integrating intelligent analysis functions and cannot detect abnormal situations.

[0039] To solve the problems in the above-mentioned related technologies, the embodiments of this application provide a security detection method and an intelligent device. A plurality of cameras are installed outside the driving device to capture environmental images outside the driving device, perform feature analysis on the captured environmental images, determine the behavior characteristics of pedestrians outside the driving device based on the extracted features, and determine whether there is a security risk for the driving device based on the behavior characteristics of the pedestrians.

[0040] The method provided by the embodiments of this application can be executed by an intelligent device, which may include a driving device or may also include an electronic device. The electronic device may be an intelligent cockpit in the driving device. Or the electronic device may be an electronic device such as a mobile phone, a tablet computer, a wearable device, a smart bracelet, a smart watch, etc. that communicates with the driving device. The electronic device has at least image analysis functions.

[0041] The driving device in the embodiments of the present application may include an intelligent vehicle, etc. Of course, the driving device may also be a ship with privacy requirements. The driving device may refer to an automobile, such as a commercial vehicle.

[0042] The method provided in the embodiments of the present application may also be implemented by an intelligent cockpit. The intelligent cockpit may be implemented by an independent server or a server cluster composed of multiple servers. The intelligent cockpit system refers to in-vehicle devices and services with intelligence and networking capabilities, such as in-vehicle infotainment systems, instrument panels, head-up displays (HUDs), streaming rearview mirrors, voice interaction systems, and other automotive electronic systems.

[0043] The electronic systems of the intelligent cockpit may consist of a complete set of systems including a central control platform, a full-color liquid crystal instrument, a central control display, an entertainment system, an intelligent speaker, a vehicle networking module, a streaming rearview mirror, a telematics system, etc. The intelligent cockpit system may be centered around a cockpit area controller and implement the functions of the above-mentioned intelligent cockpit electronic systems on a unified software and hardware platform, and integrate systems of intelligent interaction, intelligent scenarios, and personalized services. The intelligent cockpit system may form the basis for vehicle-person interaction and vehicle-external connection. The usage scenarios of the intelligent cockpit system can generally cover all scenarios of users using the vehicle. Specifically, it may include time scenarios before, during, and after users use the vehicle. It may also include spatial scenarios of the driver, co-driver, rear passengers, relevant people or objects outside the vehicle. Compared with the past command interaction, the human-machine interaction of the intelligent cockpit can combine the usage scenarios of the vehicle and the user, and is based on image recognition, voice recognition, environmental perception, etc. To achieve a more intelligent effect that better meets user needs (such as voice-changing requirements for making calls, etc.). That is to say, the intelligent cockpit refers to in-vehicle devices and services with intelligence and networking capabilities, such as in-vehicle infotainment systems, instrument panels, head-up displays (HUDs), streaming rearview mirrors, voice interaction systems, and other automotive electronic systems.

[0044] Taking the driving device in the embodiments of the present application as a commercial vehicle as an example, cameras are arranged at different positions in different directions of the commercial vehicle, and the cameras in different directions are used to collect images of different positions around the commercial vehicle.

[0045] Taking the execution subject of this solution as the driving device as an example below, specifically, this solution may be executed by a processor in the driving device. Taking the driving device as the execution subject as an example for description below, see Figure 1 , which is a schematic flowchart of the security detection method provided in the embodiments of the present application. As an example and not a limitation, the method may include the following steps:

[0046] S101, the driving device acquires one or more first images.

[0047] Among them, the first image includes an environmental picture outside the driving device collected by at least one camera deployed on the driving device.

[0048] For example, the driving device can collect an environmental picture outside the driving device through at least one camera deployed on the driving device. The image collected by any one camera can be a picture or a video. The at least one camera on the driving device can perform image collection periodically at a preset period. The preset period can be preset, or set by the user on the driving device, or sent by the user to the driving device through an electronic device. The embodiments of the present application do not limit this.

[0049] Under the condition of meeting the requirements of information security and legal regulations, if the execution subject in the embodiments of the present application is an electronic device such as a mobile phone, in the embodiments of the present application, the driving device can send the environmental picture collected by the camera to the mobile phone, or the user can set the camera of the driving device so that the mobile phone can obtain the permission of the camera, so as to obtain the picture of the external environment collected by the camera. The embodiments of the present application do not limit this.

[0050] As an example, in the embodiments of the present application, the camera can be configured to report the collected picture of the external environment when a pedestrian is collected.

[0051] S102. The driving device performs feature analysis on each first image to obtain the behavior features of the pedestrians included in each first image.

[0052] Specifically, the driving device can perform feature analysis on each obtained first image. Through feature analysis, key information (such as the behavior features of pedestrians, etc.) in each first image can be extracted. The behavior features can include information such as the actions, postures, and directions of pedestrians. These information can assist the driving device to better understand the surrounding environment and make more accurate decisions.

[0053] As an embodiment, a trained detection model is deployed in the driving device. The driving device can input each first image into the trained detection model, and the detection model can obtain the behavior features of the pedestrians corresponding to the first image through feature analysis of each input first image. Of course, the trained detection model can also be deployed in a cloud server or a mobile phone. The driving device can interact with the cloud server or the mobile phone to obtain the behavior features of the pedestrians included in each first image through the trained detection model.

[0054] S103. The driving device determines whether there is a safety risk for the driving device according to the behavior features of the pedestrians corresponding to the multiple first images.

[0055] Specifically, by analyzing the behavioral characteristics in multiple first images, the driving device can identify potential safety risks. In the embodiments of the present application, the safety risks may include whether there are safety risks to the internal privacy of the vehicle, including but not limited to behaviors such as pedestrians peeping at the driving device or staying for a long time.

[0056] S104. When there is such a safety risk, the driving device displays at least one environmental picture captured by the camera on the display screen of the driving device.

[0057] Specifically, the driving device may include a display screen (such as a central control display screen). When there is a safety risk, that is, there is a safety risk to the internal privacy of the vehicle, the driving device can display at least one environmental picture captured by the camera on the central control display screen, so as to remind the users in the vehicle to view the pictures outside the vehicle.

[0058] In a possible embodiment of the present application, when there is such a safety risk, the driving device can output a prompt message on the display screen to prompt the user to view the pictures outside the vehicle. When it is detected that the user indicates confirmation, at least one environmental picture captured by the camera is then displayed.

[0059] If S104 is executed by a mobile phone, then when an electronic device such as a mobile phone determines that there is such a safety risk, it can send an instruction to the driving device. This instruction is used to indicate that there is a safety risk to the driving device (or this instruction is used to indicate that the driving device displays at least one environmental picture captured by the camera on the display screen). For the driving device, when receiving the instruction, it can display at least one environmental picture captured by the camera on the display screen.

[0060] In one embodiment, when the driving device enters the safety mode, control the at least one camera to capture the environmental picture outside the driving device.

[0061] Among them, the safety mode is a special working mode designed to protect user privacy and reduce or avoid unnecessary data collection and storage. Users may hope to protect the privacy of the conversations in the vehicle in specific situations (such as in the conversation mode in the vehicle). Users can manually set the safety mode through the control interface of the device. In this mode, select at least one camera facing outside the vehicle to collect environmental pictures, so as to perform real-time analysis on the collected environmental pictures and extract pedestrian information, etc.

[0062] In one embodiment, refer to Figure 2 , which is a schematic flowchart of the process for obtaining behavioral characteristics provided by the embodiments of the present application. As Figure 2 shown, step S102 includes:

[0063] S201, Extract the first features in each of the first images.

[0064] Specifically, through feature analysis, a set of features are extracted from the first image, and these features can represent certain characteristics or attributes of the image. The extracted features are called "first features" because they are extracted from the "first image". These features can be local (such as corner points, edges) or global (such as color histograms, texture features). The first features are used for subsequent image processing tasks, such as classification, recognition, matching, etc.

[0065] S202, Determine the behavior features of the pedestrians outside the driving device according to the first features in multiple first images.

[0066] For example, the behavior features of pedestrians can include multiple categories, including walking, running, standing still, peeping, lingering, etc. Determine whether there are pedestrians near the vehicle and the category of the behavior features of the pedestrians according to the obtained first features, so as to be used for subsequent determination of whether there is a safety risk in the driving device.

[0067] In one embodiment, refer to Figure 3 , which is a schematic flowchart of feature analysis provided by the embodiment of the present application. As Figure 3 shown, step S201 includes:

[0068] S301, Preprocess each first image to obtain a second image corresponding to each first image; wherein, the preprocessing includes at least one of the following: normalization processing, normalization processing, image compression processing.

[0069] In the embodiment of the present application, after obtaining the environmental image outside the driving device, that is, the first image, it needs to be preprocessed before analyzing each first image, such as performing normalization processing, normalization processing, or compressing the image.

[0070] Among them, the first image can be a fish-eye image. A fish-eye image is an image taken with a fish-eye lens. This lens has a very wide viewing angle. The lens can capture environmental images from multiple angles and is suitable for installation around the vehicle for image capture.

[0071] Specifically, the pixel values of the fisheye image, i.e., the first image, can be adjusted to a specific range. For example, the pixel values are subtracted by the mean and divided by the standard deviation. This helps to eliminate the offset and scaling differences in the image data, making the image data more consistent (normalization processing); and the pixel values of the image can be adjusted to the range of 0 to 1. This is usually achieved by dividing the pixel values by the maximum pixel value (e.g., 255) (normalization processing). Normalization is helpful for subsequent processing and analysis, especially when using deep learning models; and the fisheye image can be compressed. For example, the image can be compressed to a size of 480×480×3 in width, height, and number of channels, which can reduce the data volume of the image to save storage space or improve transmission efficiency while trying to maintain the visual quality of the image (compression processing). Among them, the image obtained after preprocessing the first image is called the second image.

[0072] S302. Perform multi-scale feature analysis on the second image corresponding to each first image to obtain multiple second features of each second image at different scales.

[0073] In the embodiment of the present application, multi-scale feature analysis is performed on the preprocessed image (i.e., the "second image"). Multi-scale feature analysis refers to extracting features of an image at different scales (resolutions) in order to capture the details and structures of the image at different levels. Through multi-scale feature analysis, multiple features at different scales are extracted from the second image. These features can be used for subsequent image processing and analysis tasks, such as object detection, image classification, semantic segmentation, etc.

[0074] As an example, a lightweight backbone network can be used to extract features of the image. Among them, the lightweight backbone network is a neural network model designed for efficient feature extraction. These models usually achieve efficiency by reducing the number of parameters and computational complexity of the model. Common lightweight backbone networks include MobileNet, ShuffleNet, EfficientNet, etc. These networks use techniques such as depthwise separable convolution and group convolution to reduce the computational amount and model size while maintaining a high feature extraction ability.

[0075] Through the lightweight backbone network, four feature maps with different shapes are extracted from the input image. These feature maps capture the features of the image at different scales and are suitable for multi-scale image processing tasks. For example, multiple feature maps with different scales or shapes, i.e., second features, can be obtained, including those with sizes of 120×120×128, 60×60×256, 30×30×512, and 15×15×704 in width, height, and number of channels, etc.

[0076] S303. Determine the first feature corresponding to the first image corresponding to any one of the second images according to the second features of multiple different scales of any one of the second images.

[0077] In an embodiment of the present application, further analysis is performed on the feature maps of multiple different scales obtained through the lightweight backbone network, that is, the second features, to extract the latitude in the depth direction of the image, that is, richer features (first features).

[0078] In the above method, the preprocessing step ensures the quality and consistency of the image, and the multi-scale feature analysis captures the multi-scale information in the image, which can improve the accuracy of security detection.

[0079] In one embodiment, the first feature includes multiple first sub-features. The first feature includes multiple first sub-features, and step S303 includes:

[0080] Perform multi-channel feature analysis on the second features of each scale to obtain the first sub-features corresponding to the second features of each scale.

[0081] In an embodiment of the present application, after the feature maps of each scale pass through multiple convolutional layers and activation functions, the second features are obtained. The second features usually refer to the feature maps after preliminary processing, and these feature maps already contain certain high-level information. The purpose of multi-channel feature analysis is to extract more useful sub-features from the second features of each scale.

[0082] As an example, a bidirectional feature pyramid network can be used to further extract features from the feature maps from the backbone network, that is, the second features, to obtain feature maps with the same scale but different numbers of channels. Among them, the bidirectional feature pyramid network is an improved feature pyramid network, which enhances the representation ability of multi-scale features through a bidirectional feature fusion mechanism.

[0083] Specifically, for the above-mentioned multi-scale feature maps obtained, such as for a feature map with a size of 120×120×128 in width, height, and number of channels, use the bidirectional feature pyramid multi-channel for the bidirectional feature fusion mechanism, such as performing feature extraction from the low layer to the high layer or from the high layer to the low layer, and then using a weighting mechanism to adjust the contributions of different feature maps, etc. Finally, obtain features with the same scale as the second features but different numbers of channels, that is, the first sub-features.

[0084] Perform the same operation on the second features of each of the above scales using the bidirectional feature pyramid to obtain the first sub-features corresponding to each second feature. First sub-features with sizes of 120×120×64, 60×60×64, 30×30×64, and 15×15×64 in width, height, and number of channels can be obtained. Among them, the four first sub-features can be collectively referred to as the first feature.

[0085] In the above method, through multi-channel feature analysis, multiple first sub-features can be extracted from the second features at each scale. These sub-features can capture various attributes of the image, thereby generating a richer and more accurate feature representation. This multi-scale and multi-channel feature analysis method significantly improves the discrimination ability, robustness, and detection accuracy of the features, making the performance of the model more reliable and effective in complex scenarios.

[0086] In one embodiment, the first feature includes multiple first sub-features. Refer to Figure 4 , which is a schematic diagram of the process for determining privacy security provided by the embodiments of the present application. Figure 1 , such as Figure 4 shown, the method further includes:

[0087] S401, performing feature fusion on the multiple first sub-features to obtain a fused feature.

[0088] In the embodiments of the present application, the first sub-feature is a sub-feature extracted from the environmental image outside the driving device, that is, the first image. The sizes and numbers of channels corresponding to each first sub-feature may be different. Performing feature fusion on the multiple first sub-features to obtain a fused feature is a common multi-modal or multi-level feature fusion process. This process can significantly improve the performance of the model because it can combine information from multiple sources or different levels.

[0089] As an example, in order to fuse the above multiple first sub-features with sizes of 120×120×64, 60×60×64, 30×30×64, and 15×15×64 in width, height, and number of channels respectively, their sizes need to be unified first. Usually, bilinear interpolation is used for upsampling to adjust the sizes of all feature maps to the same size. Assuming the target size is 120×120×64 in width, height, and number of channels, each first sub-feature needs to be upsampled to 120×120×64 in width, height, and number of channels using bilinear interpolation. In this way, the sizes of all feature maps are unified to 120×120×64 in width, height, and number of channels. Then, the multiple first sub-features can be fused through element-wise addition, element-wise averaging, concatenation, and weighted fusion, etc., to obtain a fused feature.

[0090] S402, determining the behavior characteristics of the pedestrians outside the driving device according to the fused feature.

[0091] In the embodiments of the present application, the behavior characteristics of pedestrians can include multiple categories, including walking, running, standing still, lingering, etc. According to the obtained fused feature, it is determined whether there are pedestrians near the vehicle and the category of the behavior characteristics of the pedestrians, so as to be used for subsequent determination of whether there is a security threat to the privacy inside the vehicle.

[0092] As an example, a Convolutional Neural Network (CNN) model can be trained with a training dataset including images or video data outside the driving device and corresponding behavior labels of pedestrians. The dataset should contain various behavior features such as walking, running, standing still, waving, peeping, lingering, etc. Through multiple trainings, a trained Artificial Intelligence (AI) model is obtained. In the subsequent application detection process, the fused features obtained by analyzing the external environment image of the driving device are input into the AI model, and the pedestrian behavior features are automatically recognized.

[0093] In the above method, by fusing multiple first sub-features, more comprehensive and accurate fused features can be generated. Judging the behavior features of pedestrians outside the driving device based on the fused features can improve the accuracy and reliability of behavior recognition. Finally, detecting whether there is a safety risk in the driving device according to the behavior features of pedestrians can effectively protect the privacy inside the vehicle and improve the safety and user experience of the vehicle.

[0094] In one embodiment, refer to Figure 5 , which is a schematic flow chart for determining safety risks provided by an embodiment of the present application Figure 2 , as Figure 5 shown, the method further includes:

[0095] S501. Perform feature analysis on each of the first sub-features to obtain a first feature analysis result corresponding to each of the first sub-features.

[0096] In the embodiment of the present application, feature analysis is performed on the multiple first sub-features obtained through bidirectional feature pyramid analysis to obtain the analysis result (first feature analysis result) of each first sub-feature. This can be achieved through a pre-trained model or a specific algorithm.

[0097] S502. Determine a second feature analysis result of the external environment of the driving device according to the multiple first feature analysis results.

[0098] In the embodiment of the present application, the second feature analysis result is the analysis result of the external environment of the driving device obtained by comprehensively analyzing multiple first sub-features, including the category of the object outside the driving device, such as the object being a pedestrian, a vehicle or others, and including the distance between the object and the vehicle, etc.

[0099] S503. Detect whether there is a safety risk in the driving device according to the second feature analysis result.

[0100] In the embodiments of the present application, the obtained second analysis result is further analyzed to determine whether there is a safety threat to the vehicle, and further evaluation needs to be performed according to the second analysis result.

[0101] In the above method, by analyzing each first sub-feature and synthesizing multiple first analysis results to determine the second analysis result of the external environment of the driving device, finally, whether there is a safety risk in the driving device is detected according to the second analysis result. This method significantly improves the accuracy and reliability of detection.

[0102] In one embodiment, step S503 includes:

[0103] If the second feature analysis result indicates that there is a pedestrian outside the driving device, and the distance between the pedestrian and the vehicle is less than a preset distance, and / or the staying time of the pedestrian exceeds the threshold time, it is determined that there is a safety risk to the in-vehicle privacy of the vehicle.

[0104] In the embodiments of the present application, if a pedestrian is detected outside the driving device by using a related algorithm, and a distance sensor or the like can detect the position of the pedestrian and the distance between the pedestrian and the vehicle, and when it is determined that there is a pedestrian, the staying time of the pedestrian can be calculated in a timely manner. If it is detected that the distance between the pedestrian and the vehicle is less than the preset distance, and the staying time of the pedestrian exceeds the threshold time, it is determined that there is a safety risk to the in-vehicle privacy.

[0105] In the above method, by setting three conditions: there is a pedestrian outside the driving device, the distance between the pedestrian and the vehicle is less than the preset distance, and the staying time of the pedestrian exceeds the threshold time, the system can more accurately determine whether there is a safety risk to the in-vehicle privacy of the vehicle. This method significantly improves the accuracy and reliability of detection, enhances the real-time performance and response speed, and improves the user experience.

[0106] In one embodiment, referring to Figure 6 , which is a schematic flowchart of the alarm prompt provided by the embodiments of the present application. As shown in Figure 6 , the method further includes:

[0107] S601, obtain M groups of images, each group of the images includes one or more of the first images, and M is greater than or equal to 2.

[0108] In the embodiments of the present application, as an example, any camera in the vehicle can periodically collect the first images. For example, in the embodiments of the present application, one or more first images collected by the camera in one cycle can be regarded as a group of images.

[0109] Specifically, the camera can collect a set of images within time period 1, and then collect another set of images within another time period, and so on, to obtain M sets of images. Each set of images includes one or more of the first images. The number of first images in different sets of images can be the same or different, and the embodiments of the present application do not limit this.

[0110] It should be noted that the number of first images collected by the camera within different time periods can be equal or unequal.

[0111] S602, perform feature analysis on each of the images respectively to obtain the feature analysis result corresponding to each set of the images; the feature analysis result corresponding to any set of the images is used to reflect whether there is a safety risk in the driving device.

[0112] In the embodiments of the present application, a pre-trained detection model is used to perform feature analysis on each set of images respectively to obtain the feature analysis result of each set of images. For example, the feature analysis result of the first set of images is that there is a peeping behavior of a pedestrian, the feature analysis result of the second set of images is that there is a lingering behavior, and the feature analysis result of the third set of images is that there is no pedestrian.

[0113] Among them, the feature analysis result of each set of images can be used for subsequent determination of whether there is a risk to vehicle safety.

[0114] S603, if the feature analysis results corresponding to a preset number of images among the M sets of images all indicate that the driving device has the safety risk, then output the prompt information.

[0115] As an example, the preset number can be a value greater than or equal to M / 2 and less than or equal to M.

[0116] Among them, the preset number can refer to M, that is, if all M sets of images indicate that the driving device has the safety risk, then output the prompt information. Or, the preset number can be a value greater than M / 2, that is, if the feature analysis results of at least half of the M sets of images indicate that the driving device has the safety risk, then output the prompt information.

[0117] For example, if the preset number is M, it means that if the feature analysis results corresponding to all images among the M sets of images all indicate that the driving device has the safety risk, then output the prompt information.

[0118] For example, if the preset number is a value greater than M / 2, it means that if the feature analysis results corresponding to at least half of the M sets of images all indicate that the driving device has the safety risk, then output the prompt information.

[0119] In the embodiment of the present application, it is determined whether to give an alarm prompt according to the analysis results of feature analysis of M groups (for example, 3 groups) of images. If the feature analysis results of a preset number of images among the three groups of images all indicate that there is a safety risk in the driving device, a prompt message is output. The prompt message can display the video to the users in the vehicle and give a voice prompt, so that the users can confirm the risk information in time and end the safety mode. In the above method, by obtaining M groups of first images and performing safety detection on each image, the system can more comprehensively and accurately determine whether there is a safety risk in the vehicle privacy. Only when the analysis results of all images show that there is a safety risk, the intelligent device outputs a prompt message. This method significantly improves the accuracy and reliability of detection, and enhances the real-time performance and response speed.

[0120] See Figure 7 , which is the overall schematic block diagram of the safety detection provided by the embodiment of the present application. As Figure 6 shown, the steps of its privacy safety detection are as follows:

[0121] 1) Turn on the safety mode

[0122] When users have a conversation in the vehicle, they can click the safety mode control on the mobile phone or the safety mode control on the driving device to trigger the driving device to enter the safety mode. For example, the mobile phone has an application corresponding to the driving device, and the operation interface of the application has a safety mode control, or the display interface on the central control screen of the driving device displays a safety mode control.

[0123] When the driving device enters the safety mode, the driving device starts four cameras outside the driving device to capture the environmental images outside the driving device, that is, the first images.

[0124] 2) Collect the first images

[0125] The first images are the environmental images captured by multiple cameras outside the driving device, and are transmitted to the processor of the driving device through relevant communication protocols.

[0126] 3) Feature analysis

[0127] The driving device performs feature analysis on the collected first images. First, the first images need to be preprocessed to obtain second images, and then the second images are input into the backbone network model to obtain second features with different scales. Then, each second feature with different scales is respectively passed through the bidirectional pyramid model to obtain first sub-features with the same scale but different numbers of channels. The deeper latitude of the first features can obtain richer feature information.

[0128] 4) Obtain behavioral features

[0129] The driving device detects based on the obtained multiple first sub-features to obtain information such as whether there are pedestrians near the driving device, and the posture and position of the pedestrians if there are pedestrians. During the detection process, multiple first sub-features can be fused to obtain fused features, and the fused features can be used to determine whether the pedestrians are peeping, etc.; or each first sub-feature can be detected to analyze whether there are objects near the vehicle and the type of the objects, such as if the object is a pedestrian, and the distance between the pedestrian and the vehicle, the dwell time, etc. can be analyzed.

[0130] 5) Voice alarm prompt, real-time screen display

[0131] like Figure 8 As shown, it is a schematic diagram of the safety risk provided by the embodiment of the present application. If there are pedestrians near the vehicle, and the pedestrians' behavior characteristics are close to the vehicle and stay for a long time, the real-time picture will be displayed on the central control display screen in the car, and the user will be reminded by voice (prompt information) that there is a safety risk outside the driving device. The user can decide to turn off the safety mode and end the conversation mode according to the prompt information.

[0132] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0133] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of security detection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0134] Corresponding to the safety detection method described in the above embodiment, Figure 9 This is a structural block diagram of the safety detection device provided in an embodiment of the present application. For the sake of convenience of explanation, only the parts related to the embodiment of the present application are shown.

[0135] Reference Figure 9 , the safety detection device 9 comprises:

[0136] An image acquisition module 91, configured to acquire one or more first images, where the first images include environmental images outside the driving device collected by at least one camera deployed on the driving device;

[0137] A feature analysis module 92, configured to perform feature analysis on each of the first images to obtain the behavior features of pedestrians included in each of the first images;

[0138] A risk determination module 93, configured to determine whether there is a safety risk for the driving device according to the behavior features of the pedestrians corresponding to the multiple first images;

[0139] A risk prompt module 94, configured to, when there is the safety risk, display at least one environmental image collected by the at least one camera on a display screen of the driving device.

[0140] Optionally, the feature analysis module 92 is further configured to:

[0141] Extract first features from each of the first images;

[0142] Determine the behavior features of pedestrians outside the driving device according to the first features in the multiple first images.

[0143] Optionally, the feature analysis module 92 is further configured to:

[0144] Preprocess each of the first images to obtain a second image corresponding to each of the first images; wherein, the preprocessing includes at least one of the following: normalization processing, normalization processing, and image compression processing;

[0145] Perform multi-scale feature analysis on the second image corresponding to each of the first images to obtain multiple second features of different scales of each of the second images;

[0146] Determine the first features corresponding to the first images corresponding to any of the second images according to the multiple second features of different scales of any of the second images.

[0147] Optionally, the feature analysis module 92 is further configured to:

[0148] Perform multi-channel feature analysis on the second features of each scale to obtain first sub-features corresponding to the second features of each scale.

[0149] Optionally, the feature analysis module 92 is further configured to:

[0150] Fuse the multiple first sub-features to obtain a fused feature;

[0151] Determine the behavior features of pedestrians outside the driving device according to the fused feature.

[0152] Optionally, the risk determination module 93 is further configured to:

[0153] When there is a pedestrian outside the driving device, if the behavioral characteristics of the pedestrian reflect that the distance between the pedestrian and the driving device is less than or equal to a preset distance, and / or the staying time of the pedestrian exceeds a threshold time, it is determined that there is a safety risk for the driving device.

[0154] Optionally, the risk prompt module 94 is further configured to:

[0155] When there is the safety risk, output a prompt message for prompting the user that there is a safety risk for the driving device.

[0156] Optionally, the risk prompt module 94 is further configured to:

[0157] Obtain M groups of images, each group of the images including one or more of the first images, where M is greater than or equal to 2;

[0158] Perform feature analysis on each group of the images respectively to obtain a feature analysis result corresponding to each group of the images; the feature analysis result is used to reflect whether there is a safety risk for the driving device;

[0159] If the feature analysis results corresponding to the M groups of the images all indicate that there is the safety risk for the driving device, then output the prompt message; or, if the feature analysis results corresponding to N groups of the images among the M groups of the images all indicate that there is the safety risk for the driving device, and the feature analysis results corresponding to the other M - N groups of the images except the N groups of the images indicate that there is no safety risk for the driving device, then output the prompt message, where N is greater than M - N, N is greater than or equal to 2 and less than or equal to M.

[0160] Figure 10 It is a schematic structural diagram of the intelligent device provided by the embodiment of the present application. As Figure 10 shown, the intelligent device 10 in this embodiment includes: at least one processor 100 ( Figure 10 only one is shown in the figure), a processor, a memory 101, and a computer program 102 stored in the memory 101 and executable on the at least one processor 100. When the processor 100 executes the computer program 102, the steps in any of the above-mentioned safety detection method embodiments are implemented.

[0161] The intelligent device may be a desktop computer, a notebook, a palm computer, or a processor that can be a driving device. The intelligent device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand, Figure 10This is merely an example of the intelligent device 10, and does not constitute a limitation on the intelligent device 10. It may include more or fewer components than those shown in the figure, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0162] The so-called processor 100 may be a central processing unit (CPU). This processor 100 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0163] In some embodiments, the memory 101 may be an internal storage unit of the intelligent device 10, such as the hard disk or memory of the intelligent device 10. In some other embodiments, the memory 101 may also be an external storage device of the intelligent device 10, such as a plug-in hard disk equipped on the intelligent device 10, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 101 may also include both the internal storage unit and the external storage device of the intelligent device 10. The memory 101 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of the computer program. The memory 101 may also be used to temporarily store data that has been output or will be output.

[0164] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0165] The embodiments of the present application provide a computer program product. When the computer program product runs on an intelligent device, the intelligent device can implement the steps in the above-mentioned method embodiments when executed.

[0166] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of this application, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the device / smart device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk, or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0167] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0168] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0169] In the embodiments provided in this application, it should be understood that the disclosed device / smart device and method can be implemented in other ways. For example, the device / smart device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in an electrical, mechanical, or other form.

[0170] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0171] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included within the scope of security detection of the present application.

[0172] In each embodiment of the present application, the relevant user personal information that may be involved is all processed in strict accordance with the requirements of laws and regulations, following the principles of legality, legitimacy, and necessity, for reasonable purposes based on business scenarios, for the personal information actively provided by users during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with user authorization.

[0173] The user personal information processed by the applicant will vary depending on the specific product / service scenario, and it is necessary to be subject to the specific scenario of the user's use of the product / service. It may involve the user's account information, device information, driving information, vehicle information, or other relevant information. The applicant will treat the user's personal information and its processing with a high degree of diligence.

[0174] The applicant attaches great importance to the security of user personal information and has taken security protection measures that meet industry standards and are reasonable and feasible to protect the user's information and prevent personal information from being accessed, publicly disclosed, used, modified, damaged, or lost without authorization.

Claims

1. A safety detection method, characterized in that: The method comprises: Acquire one or more first images, where the first images include an environment picture outside the driving device captured by at least one camera deployed on the driving device; Performing feature analysis on each of the first images to obtain behavior features of pedestrians included in each of the first images; determining whether the driving device has a safety risk according to the behavior characteristics of the pedestrians corresponding to the plurality of first images; In the event of the safety risk, an environmental image captured by at least one of the cameras is displayed on the display screen of the driving device.

2. The method according to claim 1, characterized in that The method further comprises: When the driving device enters the safety mode, the at least one camera is controlled to capture the environment image outside the driving device.

3. The method according to claim 1, characterized in that The method further comprises: In the case where the safety risk exists, a prompt message is output, where the prompt message is used to prompt the user that the driving device has a safety risk.

4. The method according to claim 3, characterized in that The method further comprises: Acquire M groups of images, each group of the images includes one or more of the first images, and M is greater than or equal to 2; Performing the feature analysis on each group of the images respectively to obtain feature analysis results corresponding to each group of the images; the feature analysis results corresponding to any group of the images are used to reflect whether there is a safety risk in the driving device; If there are a preset number of images in the M groups of images and the feature analysis results corresponding to the images all indicate that the driving device has the safety risk, the prompt information is output.

5. The method according to claim 1, characterized in that The determining whether the driving device has a safety risk according to the behavior characteristics of the pedestrians corresponding to the plurality of first images includes: When there are pedestrians around the driving device, if the behavior characteristics of the pedestrians reflect that the distance between the pedestrians and the driving device is less than or equal to a preset distance, and / or the pedestrians' stay time exceeds a threshold time, it is determined that there is a safety risk in the driving device.

6. The method according to any one of claims 1 to 5, characterized in that: The performing feature analysis on each of the first images to obtain the behavior features of the pedestrians included in each of the first images includes: extracting a first feature from each of the first images; The behavior characteristics of the pedestrian outside the driving device are determined according to the first characteristics in the plurality of the first images.

7. The method according to claim 6, characterized in that The extracting the first feature from each of the first images comprises: Preprocessing each of the first images to obtain a second image corresponding to each of the first images; wherein the preprocessing includes at least one of the following: standardization processing, normalization processing, and image compression processing; Performing multi-scale feature analysis on the second image corresponding to each of the first images to obtain a plurality of second features of different scales for each of the second images; The first feature corresponding to the first image corresponding to any of the second images is determined according to the second features of multiple different scales of any of the second images.

8. The method according to claim 7, characterized in that The first feature includes a plurality of first sub-features; and determining the first feature corresponding to the first image corresponding to any of the second images according to the plurality of second features of different scales of any of the second images comprises: Multi-channel feature analysis is performed on the second feature at each scale to obtain a first sub-feature corresponding to the second feature at each scale.

9. The method according to claim 6, characterized in that The determining, according to the first features in the plurality of first images, the behavior features of the pedestrian outside the driving device includes: Performing feature fusion on the plurality of the first sub-features to obtain a fused feature; The behavior characteristics of the pedestrian outside the driving device are determined according to the fusion characteristics.

10. A safety detection device, characterized in that: include: An image acquisition module, used to acquire one or more first images, wherein the first image includes an environment picture outside the driving device collected by at least one camera deployed on the driving device; A feature analysis module, used to perform feature analysis on each of the first images to obtain behavior features of pedestrians included in each of the first images; a risk determination module, configured to determine whether the driving device has a safety risk according to the behavior characteristics of the pedestrians corresponding to the plurality of first images; The risk warning module is used to display the environment image captured by at least one camera on the display screen of the driving device when the safety risk exists.

11. An intelligent device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 9 is implemented.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images