In-vehicle living body prompting method, device, equipment and storage medium

Abstract

The application provides an in-vehicle living body prompting method, device, equipment and storage medium, and belongs to the technical field of intelligent automobiles. The method comprises the following steps: in response to a detection process starting, continuously acquiring video images and audio data in a vehicle; determining a living body characteristic value according to images of each frame in the video images; determining a volume parameter according to the audio data; calculating a comprehensive parameter according to the living body characteristic value and the volume parameter; if the comprehensive parameter is greater than a first preset value, controlling a loudspeaker to emit a preset sound, and executing again the step of determining the living body characteristic value to obtain a new living body characteristic value; and sending corresponding living body prompting information to a user terminal according to the new living body characteristic value and the comprehensive parameter. The method solves the problem of low accuracy of in-vehicle living body judgment.

Description

Technical Field

[0001] This application relates to the field of intelligent vehicle technology, and in particular to an in-vehicle liveness detection method, device, equipment, and storage medium. Background Technology

[0002] With the continuous development of industrial technology, automobiles have become an important means of transportation for the public. To avoid infants and young children suffering from heatstroke due to being locked in a car, it is necessary to ensure that there are no infants or young children inside the vehicle after locking it.

[0003] Currently, existing technologies typically use facial recognition to determine if there are infants or young children inside the vehicle.

[0004] However, the inventors discovered that the existing technology has at least the following technical problems: the accuracy of facial recognition judgment is poor. Summary of the Invention

[0005] This application provides a method, device, equipment, and storage medium for in-vehicle liveness detection, in order to solve the problem of poor accuracy in facial recognition for determining liveness.

[0006] In a first aspect, this application provides an in-vehicle liveness detection method, comprising: continuously acquiring video images and audio data inside the vehicle in response to the start of a detection process; determining liveness feature values ​​based on each frame of the video images; determining volume parameters based on the audio data; calculating a comprehensive parameter based on the liveness feature values ​​and the volume parameter; if the comprehensive parameter is greater than a first preset value, controlling the speaker to emit a preset sound, and re-performing the step of determining the liveness feature values ​​to obtain new liveness feature values; and sending a corresponding liveness detection message to a user terminal based on the new liveness feature values ​​and the comprehensive parameter.

[0007] In one possible implementation, determining liveness feature values ​​based on images from each frame of a video image includes: extracting the reflection image of a preset region in each frame, where the preset region is the area where the reflective object is located in each frame; obtaining the number of pixels in the preset region and the brightness of each frame; determining the liveness parameters corresponding to the target frame based on the reflection image of the target frame, the reflection images of adjacent frames, and the number of pixels in the preset region, where the target frame is any frame among all frames, and adjacent frames are frames adjacent to the target frame; inputting the reflection image of the target frame into a preset liveness classification model to obtain the target confidence score; determining the brightness parameter of the target frame based on the brightness of the target frame and the brightness of adjacent frames; and calculating the liveness feature value based on the liveness parameter, brightness parameter, and target confidence score of the target frame.

[0008] In one possible implementation, the liveness parameters corresponding to the target frame are determined based on the reflected image of the target frame, the reflected images of adjacent frames, and the number of pixels in a preset region. This includes: extracting image features of the reflected images in each frame using a preset model; dividing the image features of each frame by the number of pixels to obtain the unit image features of each frame; subtracting the unit image features of the target frame from the unit image features of adjacent frames to obtain the unit feature difference; and setting the liveness parameters corresponding to the target frame to 1 if the unit feature difference is greater than a preset difference value, otherwise setting the liveness parameters corresponding to the target frame to 0.

[0009] In one possible implementation, the brightness parameter of the target frame is determined based on the brightness of the target frame and the brightness of adjacent frames, including: subtracting the brightness of the target frame from the brightness of adjacent frames to obtain a brightness difference value. If the brightness difference value is greater than a preset brightness difference threshold, the brightness parameter of the target frame is set to 0; otherwise, the brightness parameter of the target frame is set to 1.

[0010] In one possible implementation, the liveness feature value is calculated based on the liveness parameters, brightness parameters, and target confidence of the target frame. This includes inputting the liveness parameters, brightness parameters, and target confidence of the target frame into the following formula to obtain the liveness feature value:

[0011] C image =w f ·C1·C f +w m ·C m

[0012] In the formula, C image C represents the liveness characteristic value, C1 represents the brightness parameter, C f Represents in vivo parameters, C m w represents the target confidence level. f and w m These are the preset weighting coefficients.

[0013] In one possible implementation, based on the new liveness feature value and comprehensive parameters, a corresponding liveness alert message is sent to the user terminal, including: if the new liveness feature value is greater than or equal to a preset liveness threshold, a preset high-reliability liveness alert message is sent to the user terminal. If the new liveness feature value is less than the preset liveness threshold, and the comprehensive parameters are greater than or equal to a second preset value, a preset medium-reliability liveness alert message is sent to the user terminal, wherein the second preset value is greater than a first preset value. If the new liveness feature value is less than the preset liveness threshold, and the comprehensive parameters are less than the second preset value, a preset low-reliability liveness alert message is sent to the user terminal.

[0014] In one possible implementation, determining the volume parameter based on the audio data includes: extracting the volume features of the audio data. If the volume features are greater than a preset volume threshold, the volume parameter is set to 1; otherwise, it is set to 0.

[0015] In one possible implementation, before continuously acquiring video images and audio data from inside the vehicle in response to the start of the detection process, the method further includes: acquiring an image of the vehicle interior in response to the vehicle locking process; capturing a reflection image of a preset region within the image of the vehicle interior; extracting actual features from the reflection image; calculating the target distances between the actual features and multiple pre-stored existing features; and starting the detection process if each target distance is greater than a preset distance threshold.

[0016] In one possible implementation, a comprehensive parameter is calculated based on the liveness characteristic value and volume parameter, including: inputting the liveness characteristic value and volume parameter into the following formula to obtain the comprehensive parameter:

[0017] C is =w image ·C image +w sound ·C sound

[0018] In the formula, C is C represents the comprehensive parameter. image C represents the characteristic value of a living organism. sound This represents the volume parameter, w image and w sound This represents a preset constant.

[0019] Secondly, this application provides an in-vehicle liveness detection device, comprising: a data acquisition module, used to continuously acquire video images and audio data inside the vehicle in response to the start of the detection process; a feature value determination module, used to determine liveness feature values ​​based on the images of each frame in the video images; a volume determination module, used to determine volume parameters based on the audio data; a parameter calculation module, used to calculate a comprehensive parameter based on the liveness feature values ​​and the volume parameter; a feature value acquisition module, used to control the speaker to emit a preset sound if the comprehensive parameter is greater than a first preset value, and to execute the step of determining the liveness feature value again to obtain a new liveness feature value; and a detection sending module, used to send corresponding liveness detection information to a user terminal based on the new liveness feature value and the comprehensive parameter.

[0020] Thirdly, this application provides an electronic device, including: a processor and a memory communicatively connected to the processor. The memory stores computer-executable instructions. The processor executes the computer-executable instructions stored in the memory, causing the processor to perform the in-vehicle liveness prompting method as described in the first aspect.

[0021] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the in-vehicle liveness alerting method as described in the first aspect.

[0022] The in-vehicle liveness detection method, device, equipment, and storage medium provided in this application continuously acquire video images and audio data inside the vehicle after the detection process begins. Based on the images in each frame of the video image, liveness feature values ​​are determined, and volume parameters are obtained from the audio data. Combining the liveness feature values ​​and volume parameters, a comprehensive parameter is calculated. If the comprehensive parameter is greater than a first preset value, the speaker is controlled to emit a preset sound, and the step of determining the liveness feature value is executed again to obtain a new liveness feature value. The corresponding prompt information is sent based on the new liveness feature value. This achieves the goal of determining whether there is a liveness by combining video and audio, increasing the accuracy of the judgment. Since the speaker is also controlled to emit a preset sound when the comprehensive parameter is greater than the first preset value, the range of movement of the liveness is increased when there is a liveness, which also increases the accuracy of the judgment. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0024] Figure 1 This is a schematic diagram illustrating an application scenario of the in-vehicle liveness detection method provided in this application embodiment;

[0025] Figure 2 A flowchart illustrating the in-vehicle liveness alert method provided in an embodiment of this application;

[0026] Figure 3 This is a schematic diagram of the structure of the in-vehicle liveness detection device provided in an embodiment of this application;

[0027] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0028] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0029] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0030] With the continuous development of industrial technology, cars have become an important means of transportation. However, when families with children or pets go out, they need to avoid leaving children or pets in the car, which could lead to heatstroke.

[0031] Therefore, there are technologies that use facial recognition to identify whether there are children or pets inside a car, but their accuracy is low.

[0032] To address the aforementioned technical problems, the inventors propose the following technical concept: Liveness feature values ​​are obtained from in-vehicle video, volume parameters are determined from in-vehicle audio, and a comprehensive parameter is calculated by combining the liveness feature value and the volume parameter. If the comprehensive parameter exceeds a preset value, a preset sound is played, and the liveness feature value is determined again. Based on the new liveness feature value and the comprehensive parameter, a corresponding prompt message is sent to the user terminal.

[0033] This application is applied to scenarios involving in-vehicle liveness detection. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0034] Figure 1 This is a schematic diagram illustrating an application scenario of the in-vehicle liveness detection method provided in this application embodiment. For example... Figure 1 The scene includes: camera unit 101, processing unit 102, audio unit 103 and data transmission unit 104.

[0035] In the specific implementation process, the camera unit 101 can be a camera, a vehicle recorder, etc., used to send video images inside the vehicle to the processing unit 102. The video images inside the vehicle can include video images from the interior mirror, which can be a reflective surface that can reflect images of the rear seats, and can be a metal strip, decoration, etc.

[0036] The audio unit 103 may include a microphone, a speaker, etc., for collecting audio inside the vehicle and sending the audio data to the processing unit 102, receiving the audio sent by the processing unit 102, and playing the received audio.

[0037] The processing unit 102 can be a CPU (central processing unit), a programmable logic device (PLD), a control board, an ECU (electronic control unit), etc. It processes the data obtained from the camera unit 101 and the audio unit 103, obtains the corresponding prompt information, and inputs the prompt information into the sending unit 104, causing the sending unit 104 to send the prompt information to the user terminal.

[0038] The transmitting unit 104 can be an antenna, a smart vehicle terminal (TBOX, TelematicBOX), etc., used to transmit signals to the user terminal.

[0039] The camera unit 101, audio unit 103, and data transmission unit 104 can be connected to the processing unit 102 via a wired connection or a wireless network. The wireless network connection can include various types of wired and wireless networks, such as, but not limited to: the Internet, local area network, Wireless Fidelity (WIFI), Wireless Local Area Networks (WLAN), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), 2G / 3G / 4G / 5G cellular networks, satellite communication networks, etc.

[0040] It is understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on the in-vehicle liveness detection method. In other feasible embodiments of this application, the above architecture may include more or fewer components than illustrated, or combine some components, or split some components, or arrange different components, which can be determined according to the actual application scenario and is not limited here. Figure 1 The components shown can be implemented by hardware, software, or a combination of both.

[0041] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0042] Figure 2 This is a flowchart illustrating the in-vehicle liveness detection method provided in an embodiment of this application. The executing entity of this embodiment may be... Figure 1 The processing unit 102 in the text can also be a server used by an automobile manufacturer. For example... Figure 2 As shown, the method includes:

[0043] S201: In response to the start of the detection process, continuously acquire video images and audio data inside the vehicle.

[0044] In one possible implementation, the detection process begins either when the vehicle is detected as off and locked, upon receiving a detection command from the user terminal, or when the distance between the car key and the car exceeds a preset distance. Acquiring video and audio data from inside the vehicle can be achieved by controlling a camera unit to capture video images and controlling an audio unit to capture audio data.

[0045] When acquiring audio data, if it is possible to acquire the audio data of a selected audio unit in the vehicle, it can be to acquire the audio data of a preset target audio unit. The target audio unit can be the audio unit corresponding to the area to be detected, such as the audio unit in the rear of the vehicle.

[0046] S202: Determine the liveness feature value based on the images of each frame in the video image.

[0047] In this step, the images of each frame can be input into a preset model to obtain the corresponding liveness feature values. Alternatively, the liveness feature values ​​in this step can be the average liveness feature value calculated from the liveness feature values ​​of each frame. Another approach is to use a preset program to extract features from each frame, calculate the feature difference between adjacent frames, and then calculate the liveness feature value from the feature difference.

[0048] S203: Determine the volume parameters based on the audio data.

[0049] In this step, the average volume of the audio data can be determined as the volume parameter, the maximum volume of the audio data can be determined as the volume parameter, or the median volume of the audio data can be determined as the volume parameter.

[0050] S204: Calculate the comprehensive parameters based on the in vivo characteristic values ​​and volume parameters.

[0051] In this step, the liveness feature values ​​and volume parameters can be input into a preset neural network model to obtain comprehensive parameters, or the liveness feature values ​​and volume parameters can be input into a preset formula to obtain comprehensive parameters.

[0052] In one possible implementation, a comprehensive parameter is calculated based on the liveness characteristic value and volume parameter, including:

[0053] Input the liveness characteristic value and volume parameter into the following formula to obtain the comprehensive parameter:

[0054] C is =w image ·C image +w sound ·C sound

[0055] In the formula, C is C represents the comprehensive parameter. image C represents the characteristic value of a living organism. sound This represents the volume parameter, w image and w sound This represents a preset constant.

[0056] In the formula, w image and w sound It can be a preset constant greater than 0, w image and w sound The sum of these can be equal to 1, representing the weighting coefficients of the liveness feature value and the volume parameter, respectively.

[0057] S205: If the comprehensive parameter is greater than the first preset value, control the speaker to emit a preset sound, and execute the step of determining the living characteristic value again to obtain a new living characteristic value.

[0058] In this step, the first preset value can be preset by the automobile manufacturer, and the specific value of the preset value is not specifically limited in this embodiment. Controlling the speaker to emit a preset sound can be achieved by inputting a preset electrical signal to the speaker to make the speaker emit the corresponding sound. The step of determining the living characteristic value can be step S202 as described above.

[0059] The preset sounds can be pre-set by the car manufacturer or recorded by the user. Using preset sounds can increase the range of motion of a living being in the presence of a live being.

[0060] For example, if the comprehensive parameter obtained in step S204 is 0.8 and the first preset value is 0.6, then the speaker is controlled to play the user-recorded sound, and step S202 is re-executed to obtain a new liveness characteristic value of 0.85. As another example, if the comprehensive parameter obtained in step S204 is 0.7 and the first preset value is 0.55, then the speaker is controlled to play the user-recorded sound, and step S202 is re-executed to obtain a new liveness characteristic value of 0.4.

[0061] S206: Based on the new liveness feature value and comprehensive parameters, send the corresponding liveness alert information to the user terminal.

[0062] In this step, a corresponding liveness alert message may be sent to the user terminal based on the preset liveness feature value range to which the new liveness feature value belongs and the preset comprehensive parameter range to which the comprehensive parameter belongs.

[0063] There can be at least two preset liveness feature value ranges and at least two preset comprehensive parameter ranges. There can be a correspondence between the preset liveness feature value ranges, the preset comprehensive parameter ranges, and the liveness alert information. All preset liveness feature value ranges, preset comprehensive parameter ranges, and their correspondences can be calibrated empirically.

[0064] For example, if a new liveness feature value belongs to a preset liveness feature value range A, and the comprehensive parameter belongs to a preset comprehensive parameter range c, then the corresponding liveness alert information for the preset liveness feature value range A and the preset comprehensive parameter range c will be searched. This application embodiment does not impose specific limitations on the specific range values; the correspondence between the preset liveness feature value range, the preset comprehensive parameter range, and the liveness alert information can be in tabular or dictionary format.

[0065] As described in the above embodiments, this application embodiment continuously acquires video images and audio data inside the vehicle after the detection process begins. Based on the images of each frame in the video image, it determines the liveness feature value, obtains the volume parameter from the audio data, and calculates a comprehensive parameter by combining the liveness feature value and the volume parameter. If the comprehensive parameter is greater than a first preset value, it controls the speaker to emit a preset sound and executes the step of determining the liveness feature value again to obtain a new liveness feature value. The new liveness feature value sends a corresponding prompt message, thereby realizing the determination of whether there is a live body by combining video and audio, increasing the accuracy of the judgment. Since the speaker is also controlled to emit a preset sound when the comprehensive parameter is greater than the first preset value, it increases the range of movement of the live body when there is one, which also increases the accuracy of the judgment.

[0066] In one possible implementation, step S202 above, determining the liveness feature value based on the images of each frame in the video image, specifically includes:

[0067] S2021: Extract the reflection image of a preset region in each frame, where the preset region is the area where the reflective object is located in each frame.

[0068] In this step, the reflection image of a preset region in each frame is extracted. This can be achieved by extracting the image of the preset region from each frame to obtain the reflection image. The extraction method can be to collect pixels within the preset region.

[0069] The preset area can be user-defined. User-defined areas can be defined on the terminal device and sent to the car manufacturer's server, which then sends the defined area to the car. When the executing entity is the car manufacturer's server, it is not necessary to send the preset area to the car.

[0070] S2022: Obtain the number of pixels in the preset area and the brightness of each frame.

[0071] In this step, the number of pixels in the preset area can be obtained using a pre-configured program or script, and the brightness of each frame can be the average brightness of the pixels in the preset area in each frame.

[0072] Specifically, when the image is an RGB (Red Green Blue) image, a preset program or script is used to transform the image to the HSL (Hue, Saturation, Lightnes) space, and the average value of the brightness channel across all pixels is used as the representation of the image brightness; when the image is an infrared image, the average value of all pixels can be directly used as the representation of the brightness.

[0073] S2023: Determine the liveness parameters corresponding to the target frame based on the reflection image of the target frame, the reflection images of adjacent frames, and the number of pixels in the preset area. The target frame is any frame among all frames, and the adjacent frames are the frames adjacent to the target frame.

[0074] In this step, the reflected image of the target frame, the reflected images of adjacent frames, and the number of pixels in a preset region can be input into a pre-trained neural network model to obtain the liveness parameters corresponding to the target frame. Alternatively, a preset model can be used to calculate the image features of the reflected images of the target frame and adjacent frames, and the difference between the image features corresponding to the target frame and the image features of the adjacent frames can be calculated to obtain the feature difference. The feature difference is then divided by the number of pixels to obtain the unit feature difference. If the unit feature difference is greater than a preset difference value, the liveness parameter corresponding to the target frame is set to 1; otherwise, the liveness parameter corresponding to the target frame is set to 0.

[0075] The frame adjacent to the target frame can be the frame before or after the target frame.

[0076] S2024: Input the reflection image of the target frame into the preset liveness classification model to obtain the target confidence score.

[0077] In this step, the preset liveness classification model can be trained using accurate data, and can be a neural network model or other classification or recognition model. After inputting the reflection image into the liveness classification model, multiple confidence scores corresponding to the classification results can be obtained. The confidence score corresponding to the classification result with the highest confidence score can be determined as the target confidence score. The target confidence score can be between 0 and 1.

[0078] For example, if the reflection image of the target frame is input into a preset liveness classification model, and the confidence scores corresponding to the five classification results are 0.31, 0.56, 0.23, 0.72, and 0.41, then 0.72 is taken as the target confidence score. This application does not impose specific limitations on the number of confidence scores or the specific numerical values ​​of the confidence scores.

[0079] S2025: Determine the brightness parameters of the target frame based on the brightness of the target frame and the brightness of adjacent frames.

[0080] In this step, the brightness of the target frame can be subtracted from the brightness of the adjacent frame to obtain the brightness difference, and the brightness difference can be used as the brightness parameter of the target frame.

[0081] In one possible implementation, this step involves determining the brightness parameters of the target frame based on the brightness of the target frame and the brightness of adjacent frames, specifically including:

[0082] S20251: The brightness difference value is obtained by subtracting the brightness of the target frame from the brightness of the adjacent frame.

[0083] In this step, the difference between the brightness of the target frame and the brightness of the adjacent frames can be calculated by subtracting the smaller brightness from the larger brightness, or by taking the absolute value after subtracting in any order.

[0084] For example, if the brightness of the target frame is 80 and the brightness of the adjacent frame is 50, then the brightness difference is 30. For another example, if the brightness of the target frame is 40 and the brightness of the adjacent frame is 60, then the brightness difference is 20. For yet another example, if the brightness of the target frame is 30 and the brightness of the adjacent frame is 35, then the brightness difference is 5.

[0085] S20252: If the brightness difference is greater than the preset brightness difference threshold, the brightness parameter of the target frame is set to 0; otherwise, the brightness parameter of the target frame is set to 1.

[0086] In this step, the preset brightness difference threshold can be pre-calibrated experimentally.

[0087] For example, if the brightness difference is 30 and the preset brightness difference threshold is 40, then the brightness parameter of the target frame is set to 1. As another example, if the brightness difference is 70 and the preset brightness difference threshold is 50, then the brightness parameter of the target frame is set to 0. Yet another example, if the brightness difference is 40 and the preset brightness difference threshold is 55, then the brightness parameter of the target frame is set to 1.

[0088] S2026: Calculate the liveness feature value based on the liveness parameters, brightness parameters, and target confidence of the target frame.

[0089] In this step, the liveness parameters, brightness parameters, and target confidence of the target frame can be input into a pre-trained liveness feature value calculation model to obtain liveness feature values, or the liveness parameters, brightness parameters, and target confidence of the target frame can be input into a preset formula to obtain liveness feature values.

[0090] In one possible implementation, in this step, the liveness feature value is calculated based on the liveness parameters, brightness parameters, and target confidence of the target frame, specifically including:

[0091] Input the liveness parameters, brightness parameters, and target confidence of the target frame into the following formula to obtain the liveness feature value:

[0092] C image =w f ·C1·C f +w m ·C m

[0093] In the formula, C image C represents the liveness characteristic value, C1 represents the brightness parameter, C f Represents in vivo parameters, C m w represents the target confidence level. f and w m These are the preset weighting coefficients. f and w m w can be a preset constant greater than 0. f and w m The sum can be 1.

[0094] As described in the above embodiments, this application embodiment extracts the reflection image of a preset area and obtains the number of pixels in the preset area and the brightness of each frame. Based on the reflection image of the target frame, the reflection images of adjacent frames, and the number of pixels in the preset area, the liveness parameters corresponding to the target frame are obtained. A liveness classification model is used to determine the target confidence level corresponding to the reflection image of the target frame. The brightness parameters of the target frame are determined based on the brightness of the target frame and the brightness of adjacent frames. Liveness feature values ​​are calculated based on the liveness parameters, brightness parameters, and target confidence level of the target frame. Because a reflection image is used, it is possible to determine whether there are live bodies in the obscured areas of the back row. Furthermore, it considers changes in brightness, avoiding the situation where large changes in brightness lead to larger liveness parameters and misjudgments, thus increasing the accuracy of the judgment. The combination of liveness feature values, brightness parameters, and target confidence level further enhances the accuracy of the judgment.

[0095] In one possible implementation, in step S2023 above, the liveness parameters corresponding to the target frame are determined based on the reflected image of the target frame, the reflected images of adjacent frames, and the number of pixels in a preset region, including:

[0096] S20231: Use a preset model to extract the image features of the reflected images in each frame.

[0097] In this step, the model for extracting image features can be a deep learning image backbone network model, such as VGG (Visual Geometry Group Network), ResNet (Residual Network), EfficientNet, etc. The model can be pre-trained using experimental data.

[0098] S20232: Divide the image features of each frame by the number of pixels to obtain the unit image features of each frame.

[0099] In this step, for example, if the image feature of a certain frame is 30 and the number of pixels is 300, then the unit image feature is 0.1. This application embodiment does not impose specific limitations on the specific values ​​of image features and the number of pixels. Image features may also have directional attributes, and correspondingly, the calculated unit image feature also has directional attributes.

[0100] S20233: Subtract the unit feature features of the target frame from the unit feature features of the adjacent frames to obtain the unit feature difference.

[0101] In this step, the difference can be calculated by subtracting a smaller unit image feature from a larger one. Alternatively, it can be calculated by subtracting one unit image feature from another, taking the absolute value of the result, and obtaining the unit feature difference.

[0102] For example, if the unit frame feature of the target frame is 0.5 and the unit frame feature of the adjacent frame is 0.6, then the unit feature difference can be 0.1. If the unit frame feature of the target frame is 0.3 and the unit frame feature of the adjacent frame is 0.7, then the unit feature difference is 0.4.

[0103] S20234: If the unit feature difference is greater than the preset difference, then set the liveness parameter corresponding to the target frame to 1; otherwise, set the liveness parameter corresponding to the target frame to 0.

[0104] In this step, the preset difference can be pre-calibrated based on the experiment.

[0105] For example, if the unit feature difference is 0.5 and the preset difference is 0.7, then the liveness parameter corresponding to the target frame is set to 0; for another example, if the unit feature difference is 0.4 and the preset difference is 0.55, then the liveness parameter corresponding to the target frame is set to 0; and for yet another example, if the unit feature difference is 0.7 and the preset difference is 0.4, then the liveness parameter corresponding to the target frame is set to 1.

[0106] As can be seen from the description of the above embodiments, the embodiments of this application extract the image features of the reflected image in each frame, divide the image features of each frame by the number of pixels to obtain the unit image features of each frame, obtain the unit feature difference from the unit image features of the target frame and the adjacent frames, and set the liveness parameter corresponding to the target frame to 1 when the unit feature difference is greater than a preset difference value, and set the liveness parameter corresponding to the target frame to 0 in other cases. This realizes the assignment of liveness parameters when the unit feature difference is large, which is convenient for subsequent calculation of comprehensive parameters. Since the unit feature difference is used, it can more accurately determine whether there is a liveness.

[0107] Meanwhile, by employing image backbone network models such as VGG to extract image features from reflected images, good robustness can be achieved.

[0108] In one possible implementation, in S206 above, based on the new liveness feature value and comprehensive parameters, a corresponding "liveness detected" message is sent to the user terminal, specifically including:

[0109] S2061: If the new liveness feature value is greater than or equal to the preset liveness threshold, a preset high-reliability liveness alert message is sent to the user terminal.

[0110] In this step, the liveness threshold can be pre-calibrated experimentally. The high-reliability liveness alert message can be pre-set by the car manufacturer. The high-reliability liveness alert message can be sent via SMS, voice message, software message, official account message, mini-program message, etc.

[0111] For example, if the new liveness characteristic value is 0.7 and the liveness threshold is 0.6, then the system will send a message to the user terminal saying "A live person has been detected in the back seat (high confidence)". Another example is if the new liveness characteristic value is 0.66 and the liveness threshold is 0.5, then the system will send a message to the user terminal saying "There is a high probability that a live person has been detected in the back seat of the vehicle; please check". Yet another example is if the new liveness characteristic value is 0.9 and the liveness threshold is 0.75, then the system will send a message to the user terminal saying "There may be a child or pet in the back seat of the vehicle; please check".

[0112] S2062: If the new liveness feature value is less than the preset liveness threshold and the comprehensive parameter is greater than or equal to the second preset value, then a preset medium reliability liveness prompt message is sent to the user terminal, wherein the second preset value is greater than the first preset value.

[0113] In this step, the reliability indicator may include a liveness detection message, or it may be pre-set by the car manufacturer.

[0114] For example, if the new liveness feature value is 0.4, the liveness threshold is 0.6, the comprehensive parameter is 0.7, and the second preset value is 0.9, then the system will send "A live person has been detected in the back seat (confidence level is medium)" to the user terminal. Another example is if the new liveness feature value is 0.5, the liveness threshold is 0.55, the comprehensive parameter is 0.8, and the second preset value is 0.85, then the system will send "There is a certain probability of a live person in the back seat; please check." Yet another example is if the new liveness feature value is 0.6, the liveness threshold is 0.7, the comprehensive parameter is 0.8, and the second preset value is 0.9, then the system will send "There may be a live person in the back seat of the vehicle, with a probability of 30%-60%" to the user terminal.

[0115] S2063: If the new liveness feature value is less than the preset liveness threshold and the comprehensive parameter is less than the second preset value, then a preset low reliability liveness alert message is sent to the user terminal.

[0116] In this step, low reliability results in liveness warning messages such as "Live body detected in the back row (low confidence)", "There is a small probability of an infant or pet in the back row, please check", and "There may be a pet in the back row, with a probability of less than 30%, please confirm".

[0117] In this embodiment, "high," "medium," and "low" represent the degree of qualitative assessment and are used to name the prompt information; no specific numerical value is imposed.

[0118] As can be seen from the description of the above embodiments, the embodiments of this application send a high-reliability liveness alert to the user terminal when the new liveness feature value is greater than or equal to a preset liveness threshold; send a medium-reliability liveness alert to the user terminal when the new liveness feature value is less than the preset liveness threshold and the comprehensive parameter is greater than or equal to a second preset value; and send a low-reliability liveness alert to the user terminal when the new liveness feature value is less than the preset liveness threshold and the comprehensive parameter is less than the second preset value. This achieves the effect of providing graded reporting to users, allowing users to be more aware of the specific situation.

[0119] In one possible implementation, if the comprehensive parameter is less than or equal to a first preset value, a prompt message indicating that no live body was detected in the back row is sent to the user terminal.

[0120] In one possible implementation, step S203 above, determining the volume parameter based on the audio data, specifically includes:

[0121] S2031: Extract volume features from audio data.

[0122] In this step, you can extract any one of the following from the audio data: maximum volume, median volume, or average volume.

[0123] S2032: If the volume characteristic is greater than the preset volume threshold, the volume parameter is set to 1; otherwise, it is set to 0.

[0124] In this step, the volume threshold can be pre-calibrated experimentally.

[0125] For example, if the maximum volume is 75 dB and the preset volume threshold is 50 dB, then the volume parameter is set to 1; if the maximum volume is 65 dB and the preset volume threshold is 70 dB, then the volume parameter is set to 0; and if the maximum volume is 45 dB and the preset volume threshold is 60 dB, then the volume parameter is set to 0.

[0126] As can be seen from the description of the above embodiments, the embodiments of this application compare the volume characteristics and volume threshold of audio data. When the volume characteristics are greater than the volume threshold, the volume parameter is determined to be 1, and when the volume characteristics are less than or equal to the volume threshold, the volume parameter is determined to be 0. This achieves the effect of judging whether there is a living person in the car by combining the volume inside the car, thus increasing the accuracy of the judgment.

[0127] In one possible implementation, before step S201, which continuously acquires video images and audio data from inside the vehicle in response to the start of the detection process, the method further includes:

[0128] S210: In response to vehicle door locking, acquire an image of the vehicle interior.

[0129] In this step, the camera unit can be controlled to capture images in response to the door being locked, thereby obtaining an image of the interior of the vehicle; or the camera unit can be controlled to capture images in response to receiving a vehicle locking signal, thereby obtaining an image of the interior of the vehicle.

[0130] S211: Extract the reflection image of a preset area from the in-vehicle image.

[0131] This step is similar to step S2021 above. It can be to crop the reflection image of the corresponding rear row to be detected area on the rear mirror at a fixed position in the image. Here, the mirror can be a highly reflective surface of the vehicle interior, such as a chrome trim strip.

[0132] S212: Extract the actual features from the reflection image.

[0133] In this step, the actual features in the reflected image are extracted, which can be similar to step S20231 above, using a preset model. The preset model can be a SIFT (Scale-invariant feature transform) model, a SURF (Speeded Up Robust Features) model, etc.

[0134] S213: Calculate the target distance between the actual feature and multiple pre-stored existing features. If the target distance is greater than the preset distance threshold, the detection process begins.

[0135] In this step, the Euclidean distance between the actual feature and any existing feature can be calculated to obtain the target distance. The preset distance threshold can be pre-calibrated experimentally.

[0136] The pre-stored existing features can be obtained through the following steps S2131 to S2133:

[0137] S2131: Extract reflection images under different lighting conditions. Different lighting conditions can be combinations of conditions at different times (morning, noon, evening, night, etc.) and different locations (outdoors, under trees, indoors, underground parking, streetlights, etc.).

[0138] S2132: Extract the features of the reflected image. The extraction method is the same as the method of extracting image features in step S20231 above.

[0139] S2133: Cluster all features, extract the centroids of each class and store them in the feature database. Then, according to the number of features in each class, randomly sample them according to a preset ratio and an average distribution. The sampled features are used as existing features, which can be stored in the database or storage unit.

[0140] As described in the above embodiments, this application embodiment acquires an in-vehicle image in response to vehicle door locking, extracts a reflection image of a preset area from the in-vehicle image, extracts actual features from the reflection image, calculates the target distance between the actual reflection image and multiple stored existing features, and if the target distance is greater than a preset distance threshold, the detection process begins. This involves first acquiring the in-vehicle image and comparing it with existing images. If the feature difference between the two images is too large, it is determined that there may be a new object in the back (which may be a living or non-living object). If a new object is found, subsequent determination of whether a living object exists is performed; otherwise, the subsequent process is not performed, reducing the computational load. Simultaneously, the use of machine vision technologies such as SIFT and SURF can increase the detection speed.

[0141] In one possible implementation, audio data can be acquired periodically, and the above steps S205 and S206 can be executed periodically. After a live person is detected, a preset high-reliability live person alert message can be sent to the user terminal.

[0142] Figure 3 This is a schematic diagram of the in-vehicle liveness detection device provided in an embodiment of this application. Figure 3 As shown, the in-vehicle liveness detection device 300 includes: a data acquisition module 301, a feature value determination module 302, a volume determination module 303, a parameter calculation module 304, a feature value acquisition module 305, and a detection sending module 306.

[0143] The data acquisition module 301 is used to continuously acquire video images and audio data inside the vehicle in response to the start of the detection process;

[0144] The feature value determination module 302 is used to determine the liveness feature value based on the images of each frame in the video image;

[0145] The volume determination module 303 is used to determine the volume parameters based on the audio data;

[0146] The parameter calculation module 304 is used to calculate comprehensive parameters based on the liveness characteristic values ​​and volume parameters;

[0147] The feature value acquisition module 305 is used to control the speaker to emit a preset sound if the comprehensive parameter is greater than the first preset value, and to execute the step of determining the living feature value again to obtain a new living feature value.

[0148] The prompt sending module 306 is used to send corresponding liveness prompt information to the user terminal based on the new liveness feature value and comprehensive parameters.

[0149] The apparatus provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0150] In one possible implementation, the feature value determination module 302 is specifically used to extract the reflection image of a preset region in each frame, wherein the preset region is the region where the reflective object is located in each frame; obtain the number of pixels in the preset region and the brightness of each frame; determine the liveness parameters corresponding to the target frame based on the reflection image of the target frame, the reflection images of adjacent frames, and the number of pixels in the preset region, wherein the target frame is any frame among all frames, and adjacent frames are frames adjacent to the target frame; input the reflection image of the target frame into a preset liveness classification model to obtain the target confidence score; determine the brightness parameter of the target frame based on the brightness of the target frame and the brightness of adjacent frames; and calculate the liveness feature value based on the liveness parameter, brightness parameter, and target confidence score of the target frame.

[0151] The apparatus provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0152] In one possible implementation, the feature value determination module 302 is specifically used to extract the image features of the reflected image in each frame using a preset model; divide the image features of each frame by the number of pixels to obtain the unit image features of each frame; subtract the unit image features of the target frame from the unit image features of the adjacent frames to obtain the unit feature difference; if the unit feature difference is greater than a preset difference value, then set the liveness parameter corresponding to the target frame to 1, otherwise set the liveness parameter corresponding to the target frame to 0.

[0153] The apparatus provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0154] In one possible implementation, the feature value determination module 302 is specifically used to obtain a brightness difference value by subtracting the brightness of the target frame from the brightness of the adjacent frame; if the brightness difference value is greater than a preset brightness difference threshold, the brightness parameter of the target frame is determined to be 0, otherwise the brightness parameter of the target frame is determined to be 1.

[0155] The apparatus provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0156] In one possible implementation, the feature value determination module 302 is specifically used to input the liveness parameters, brightness parameters, and target confidence of the target frame into the following formula to obtain the liveness feature value:

[0157] C image =w f ·C1·C f +w m ·C m

[0158] In the formula, C image C represents the liveness characteristic value, C1 represents the brightness parameter, C f Represents in vivo parameters, C m w represents the target confidence level. f and w m These are the preset weighting coefficients.

[0159] The apparatus provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0160] In one possible implementation, the prompt sending module 306 is specifically configured to send a preset high-reliability liveness prompt message to the user terminal if the new liveness feature value is greater than or equal to a preset liveness threshold; if the new liveness feature value is less than the preset liveness threshold and the comprehensive parameter is greater than or equal to a second preset value, send a preset medium-reliability liveness prompt message to the user terminal, wherein the second preset value is greater than a first preset value; and if the new liveness feature value is less than the preset liveness threshold and the comprehensive parameter is less than the second preset value, send a preset low-reliability liveness prompt message to the user terminal.

[0161] The apparatus provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0162] In one possible implementation, the volume determination module 303 is specifically used to extract the volume characteristics of the audio data; if the volume characteristics are greater than a preset volume threshold, the volume parameter is determined to be 1, otherwise it is determined to be 0.

[0163] The apparatus provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0164] In one possible implementation, the in-vehicle liveness alert device 300 further includes a process initiation module 307.

[0165] The process start module 307 is used to acquire an image inside the vehicle in response to the vehicle locking; capture a reflection image of a preset area in the image inside the vehicle; extract actual features from the reflection image; calculate the target distance between the actual features and multiple pre-stored existing features; if each target distance is greater than a preset distance threshold, the detection process begins.

[0166] The apparatus provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0167] In one possible implementation, the parameter calculation module 304 is specifically used to input the liveness feature value and volume parameter into the following formula to obtain the comprehensive parameter:

[0168] C is =w image ·C image +w sound ·C sound

[0169] In the formula, C is C represents the comprehensive parameter. image C represents the characteristic value of a living organism. sound This represents the volume parameter, w image and w sound This represents a preset constant.

[0170] The apparatus provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0171] To implement the above embodiments, this application also provides an electronic device.

[0172] refer to Figure 4 The diagram illustrates a structural schematic of an electronic device 400 suitable for implementing embodiments of this application. The electronic device 400 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 4 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0173] like Figure 4As shown, the electronic device 400 may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 401 and a memory 402 communicatively connected to the processor. The processor can perform various appropriate actions and processes based on programs stored in the memory 402, computer-executed instructions, or programs loaded from storage device 408 into random access memory (RAM) 403, to implement the in-vehicle liveness alerting method in any of the above embodiments. The memory may be a read-only memory (ROM). The RAM 403 also stores various programs and data required for the operation of the electronic device 400. The processing device 401, the memory 402, and the RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0174] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 An electronic device 400 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0175] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 409, or installed from storage device 408, or installed from memory 402. When the computer program is executed by processing device 401, it performs the functions defined in the methods of embodiments of this application.

[0176] It should be noted that the computer-readable storage medium described above in this application can be a computer-readable signal medium or a computer storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0177] The aforementioned computer-readable storage medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0178] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.

[0179] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0180] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0181] The modules described in the embodiments of this application can be implemented in software or in hardware. The names of the units are not necessarily limiting of the module itself; for example, a data acquisition module can also be described as an "image and audio acquisition module".

[0182] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0183] This application also provides a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the technical solution of the in-vehicle liveness alert method in any of the above embodiments. Its implementation principle and beneficial effects are similar to those of the in-vehicle liveness alert method, and can be found in the implementation principle and beneficial effects of the in-vehicle liveness alert method, which will not be repeated here.

[0184] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0185] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the technical solution of the in-vehicle liveness alert method in any of the above embodiments. Its implementation principle and beneficial effects are similar to those of the in-vehicle liveness alert method, and can be found in the implementation principle and beneficial effects of the in-vehicle liveness alert method, which will not be repeated here.

[0186] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0187] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

[0188] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0189] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for in-vehicle liveness detection, characterized in that, include: In response to the start of the detection process, it continuously acquires video images and audio data from inside the vehicle; The reflection image of a preset region is extracted from each frame of the video image, wherein the preset region is the region where the reflective object is located in each frame; Obtain the number of pixels in the preset area and the brightness of each frame; Based on the reflection image of the target frame, the reflection images of adjacent frames, and the number of pixels in the preset region, the liveness parameters corresponding to the target frame are determined, wherein the target frame is any frame among all frames, and the adjacent frames are frames adjacent to the target frame; The reflection image of the target frame is input into a preset liveness classification model to obtain the target confidence score; The brightness parameters of the target frame are determined based on the brightness of the target frame and the brightness of the adjacent frames. Calculate the liveness feature value based on the liveness parameters, brightness parameters, and target confidence level of the target frame; Based on the audio data, determine the volume parameters; Calculate the comprehensive parameters based on the liveness characteristic values ​​and the volume parameters; If the comprehensive parameter is greater than the first preset value, the speaker is controlled to emit a preset sound, and the step of determining the living feature value is executed again to obtain a new living feature value; Based on the new liveness feature value and the comprehensive parameters, a corresponding liveness alert message is sent to the user terminal.

2. The method according to claim 1, characterized in that, The step of determining the liveness parameters corresponding to the target frame based on the reflection image of the target frame, the reflection images of adjacent frames, and the number of pixels in the preset region includes: The image features of the reflected images in each frame are extracted using a preset model; Divide the image features of each frame by the number of pixels to obtain the unit image features of each frame; The difference between the unit image features of the target frame and the unit image features of the adjacent frames is obtained; If the unit feature difference is greater than the preset difference, the liveness parameter corresponding to the target frame is set to 1; otherwise, the liveness parameter corresponding to the target frame is set to 0.

3. The method according to claim 1, characterized in that, Determining the brightness parameter of the target frame based on the brightness of the target frame and the brightness of the adjacent frames includes: The brightness difference value is obtained by subtracting the brightness of the target frame from the brightness of the adjacent frame; If the brightness difference is greater than a preset brightness difference threshold, the brightness parameter of the target frame is set to 0; otherwise, the brightness parameter of the target frame is set to 1.

4. The method according to claim 1, characterized in that, The step of calculating the liveness feature value based on the liveness parameters, brightness parameters, and target confidence level of the target frame includes: The liveness parameters, brightness parameters, and target confidence of the target frame are input into the following formula to obtain the liveness feature value: In the formula, This represents the living feature value. This represents the brightness parameter. This refers to the in vivo parameters. This represents the target confidence level. and These are the preset weighting coefficients.

5. The method according to any one of claims 1 to 4, characterized in that, The step of sending a corresponding "live presence in the back row" notification to the user terminal based on the new liveness feature value and the comprehensive parameters includes: If the new liveness feature value is greater than or equal to the preset liveness threshold, a preset high-reliability liveness alert message is sent to the user terminal. If the new liveness feature value is less than the preset liveness threshold and the comprehensive parameter is greater than or equal to the second preset value, then a preset medium reliability liveness alert message is sent to the user terminal, wherein the second preset value is greater than the first preset value. If the new liveness feature value is less than the preset liveness threshold and the comprehensive parameter is less than the second preset value, then a preset low reliability liveness alert message is sent to the user terminal.

6. The method according to any one of claims 1 to 4, characterized in that, Determining the volume parameter based on the audio data includes: Extract the volume features of the audio data; If the volume characteristic is greater than the preset volume threshold, the volume parameter is set to 1; otherwise, it is set to 0.

7. The method according to any one of claims 1 to 4, characterized in that, Before the continuous acquisition of video images and audio data inside the vehicle in response to the start of the detection process, the method further includes: In response to the vehicle locking, acquire images of the interior of the vehicle; Extract the reflection image of a preset area from the in-vehicle image; Extract the actual features from the reflected image; Calculate the target distance between the actual feature and multiple pre-stored existing features. If each target distance is greater than a preset distance threshold, then the detection process begins.

8. The method according to any one of claims 1 to 4, characterized in that, The step of calculating the comprehensive parameters based on the liveness feature values ​​and the volume parameters includes: Input the liveness feature value and the volume parameter into the following formula to obtain the comprehensive parameter: In the formula, This represents the comprehensive parameter. This represents the living feature value. This indicates the volume parameter. and This represents a preset constant.

9. An in-vehicle liveness detection device, characterized in that, include: The data acquisition module is used to continuously acquire video images and audio data inside the vehicle in response to the start of the detection process; The feature value determination module is used to extract the reflection image of a preset region in each frame of the video image, wherein the preset region is the region where the reflective object is located in each frame; Obtain the number of pixels in the preset area and the brightness of each frame; Based on the reflection image of the target frame, the reflection images of adjacent frames, and the number of pixels in the preset region, the liveness parameters corresponding to the target frame are determined, wherein the target frame is any frame among all frames, and the adjacent frames are frames adjacent to the target frame; The reflection image of the target frame is input into a preset liveness classification model to obtain the target confidence score; The brightness parameters of the target frame are determined based on the brightness of the target frame and the brightness of the adjacent frames. Calculate the liveness feature value based on the liveness parameters, brightness parameters, and target confidence level of the target frame; A volume determination module is used to determine volume parameters based on the audio data; The parameter calculation module is used to calculate the comprehensive parameters based on the liveness feature values ​​and the volume parameters; The feature value acquisition module is used to control the speaker to emit a preset sound if the comprehensive parameter is greater than a first preset value, and to execute the step of determining the living feature value again to obtain a new living feature value. The notification sending module is used to send a corresponding liveness notification message to the user terminal based on the new liveness feature value and the comprehensive parameters.

10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the in-vehicle liveness alert method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the in-vehicle liveness alert method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, is used to implement the in-vehicle liveness alert method as described in any one of claims 1 to 8.

Images