Video verification method, video verification device, electronic device, and storage medium

Abstract

The embodiment of the application provides a video investigation method, a video investigation device, an electronic device and a storage medium, and belongs to the technical field of artificial intelligence, and is applied to the fields of financial technology and health care. The method comprises the following steps: acquiring an ambient light image, an audio signal, a target geomagnetic intensity and a target illumination intensity of a video investigation site in a video investigation process; performing image feature extraction on the ambient light image to obtain a light vector; performing audio feature extraction on the audio signal to obtain an acoustic feature; performing consistency checking on the video investigation site and a preset reporting site according to at least one of the light vector, the acoustic feature, the target geomagnetic intensity and the target illumination intensity, to obtain video investigation data; wherein the video investigation data is used to indicate whether the video investigation site and the preset reporting site are consistent or inconsistent. The embodiment of the application can improve the accuracy of video investigation.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and is applied to the fields of fintech and healthcare, and particularly to a video due diligence method, a video due diligence device, an electronic device, and a storage medium. Background Technology

[0002] Video due diligence refers to the process of verifying user identity and conducting background checks through remote video calls. In fintech scenarios, for example, video due diligence remotely verifies the authenticity of the borrower's declared office or business premises. In healthcare scenarios, it verifies the authenticity of the declared residential address of a medical insurance policyholder. Currently, the verification of user-declared premises in video due diligence relies entirely on manual review. Manual review depends on subjective human judgment, and inconsistent judgment standards lead to misjudgments and omissions, resulting in low accuracy. Summary of the Invention

[0003] The main objective of this application is to provide a video due diligence method, a video due diligence device, an electronic device, and a storage medium, with the aim of improving the accuracy of video due diligence.

[0004] To achieve the above objectives, a first aspect of this application proposes a video due diligence method, the method comprising: Acquire ambient light and shadow images, audio signals, target geomagnetic intensity, and target illumination intensity at the video due diligence location during the video due diligence process; Image features are extracted from the ambient light and shadow image to obtain the light and shadow vector; Audio features are extracted from the audio signal to obtain acoustic features; Based on at least one of the light and shadow vector, the acoustic features, the target geomagnetic intensity, and the target illumination intensity, a consistency check is performed on the video due diligence location and the preset reporting location to obtain video due diligence data; wherein, the video due diligence data is used to indicate whether the video due diligence location and the preset reporting location are consistent or inconsistent.

[0005] In some embodiments, the step of extracting image features from the ambient light and shadow image to obtain a light and shadow vector includes: Semantic segmentation is performed on the ambient light and shadow image to obtain fixed object regions; Feature point detection is performed on the fixed object region to obtain key points at the root of the object and key points at the top of the shadow. The light and shadow vector is calculated based on the key points at the root of the object and the key points at the top of the shadow.

[0006] In some embodiments, the light and shadow vector includes the light and shadow direction and the target light and shadow length. The consistency verification of the video due diligence location and the preset reporting location is performed based on at least one of the light and shadow vector, the acoustic features, the target geomagnetic intensity, and the target illumination intensity to obtain video due diligence data, including: Obtain the theoretical solar azimuth and theoretical solar altitude angle of the preset reporting location; Calculate the first deviation between the theoretical solar azimuth angle and the direction of light and shadow; Calculate the theoretical shadow length based on the theoretical solar altitude angle; Calculate the second deviation between the theoretical light and shadow length and the target light and shadow length; Based on at least one of the first deviation, the second deviation, the acoustic characteristics, the target geomagnetic intensity, and the target illumination intensity, the consistency of the video due diligence location and the preset reporting location is verified to obtain the video due diligence data.

[0007] In some embodiments, obtaining the theoretical solar azimuth and theoretical solar altitude angle of the preset reporting location includes: Obtain the preset reporting time and the location coordinates of the preset reporting location; Calculate the theoretical solar altitude angle based on the preset reporting time and the location coordinates; The theoretical solar azimuth angle is calculated based on the theoretical solar altitude angle, the preset reporting time, and the location coordinates.

[0008] In some embodiments, the step of verifying the consistency between the video due diligence location and the preset reporting location based on at least one of the first deviation, the second deviation, the acoustic features, the target geomagnetic intensity, and the target illumination intensity to obtain the video due diligence data includes: The acoustic features are subjected to spatial semantic verification to obtain a semantic verification result; wherein, the semantic verification result is used to indicate whether the acoustic feature verification passes or fails. Based on at least one of the first deviation, the second deviation, the semantic verification result, the target geomagnetic intensity, and the target illumination intensity, a consistency verification is performed on the video due diligence location and the preset reporting location to obtain the video due diligence data.

[0009] In some embodiments, the step of performing a consistency check on the video due diligence location and the preset reporting location based on at least one of the first deviation, the second deviation, the semantic verification result, the target geomagnetic intensity, and the target illumination intensity to obtain the video due diligence data includes: Obtain the reference geomagnetic intensity and reference light intensity of the preset reporting location; The reference geomagnetic intensity and the target geomagnetic intensity are compared to obtain a first comparison result; wherein, the first comparison result is used to indicate whether the reference geomagnetic intensity and the target geomagnetic intensity match or do not match; A second comparison result is obtained by comparing the reference light intensity and the target light intensity; wherein, the second comparison result is used to indicate whether the reference light intensity and the target light intensity match or do not match; Based on at least one of the first deviation, the second deviation, the semantic verification result, the first comparison result, and the second comparison result, a consistency verification is performed on the video due diligence location and the preset reporting location to obtain the video due diligence data.

[0010] In some embodiments, the step of performing consistency verification on the video due diligence location and the preset reporting location based on at least one of the first deviation, the second deviation, the semantic verification result, the first comparison result, and the second comparison result to obtain the video due diligence data includes: If the first deviation is less than the first preset deviation threshold, the second deviation is less than the second preset deviation threshold, the semantic verification result indicates that the acoustic feature verification is passed, the first comparison result indicates that the reference geomagnetic intensity and the target geomagnetic intensity match, and the second comparison result indicates that the reference illumination intensity and the target illumination intensity match, then it is determined that the video due diligence data is consistent with the video due diligence location and the preset reporting location; If the first deviation is greater than or equal to the first preset deviation threshold, the second deviation is greater than or equal to the second preset deviation threshold, the semantic verification result indicates that the acoustic feature verification has failed, the first comparison result indicates that the reference geomagnetic intensity and the target geomagnetic intensity do not match, or the second comparison result indicates that the reference illumination intensity and the target illumination intensity do not match, then it is determined that the video due diligence data is inconsistent between the video due diligence location and the preset reporting location.

[0011] To achieve the above objectives, a second aspect of this application provides a video due diligence apparatus, the apparatus comprising: The acquisition module is used to acquire ambient light and shadow images, audio signals, target geomagnetic intensity, and target illumination intensity of the video due diligence location during the video due diligence process. The first feature extraction module is used to extract image features from the ambient light and shadow image to obtain a light and shadow vector; The second feature extraction module is used to extract audio features from the audio signal to obtain acoustic features; The verification module is used to verify the consistency between the video due diligence location and the preset reporting location based on at least one of the light and shadow vector, the audio signal, the acoustic features, the target geomagnetic intensity, and the target illumination intensity, to obtain video due diligence data; wherein, the video due diligence data is used to indicate whether the video due diligence location and the preset reporting location are consistent or inconsistent.

[0012] To achieve the above objectives, a third aspect of the present application provides an electronic device, the electronic device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method described in the first aspect.

[0013] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0014] The video due diligence method, video due diligence device, electronic device, and computer-readable storage medium proposed in this application acquire ambient light and shadow images, audio signals, target geomagnetic intensity, and target illumination intensity of the video due diligence location during the video due diligence process. Image features are extracted from the ambient light and shadow images to obtain light and shadow vectors, and audio features are extracted from the audio signals to obtain acoustic features. Based on at least one of the light and shadow vectors, acoustic features, target geomagnetic intensity, and target illumination intensity, consistency verification is performed between the video due diligence location and the preset reporting location to obtain video due diligence data. This enables automated video due diligence, eliminates interference from subjective human judgment, and improves the accuracy of video due diligence. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram illustrating the application environment of the video due diligence method provided in the embodiments of this application; Figure 2 This is a flowchart of the video due diligence method provided in the embodiments of this application; Figure 3 yes Figure 2 The flowchart of step S220 in the text; Figure 4 yes Figure 2 The flowchart of step S240 in the text; Figure 5 yes Figure 4 The flowchart of step S410 in the middle; Figure 6 yes Figure 4 The flowchart of step S450 in the text; Figure 7 yes Figure 6 The flowchart of step S620 in the middle; Figure 8 yes Figure 7 The flowchart of step S740 in the middle; Figure 9 This is a schematic diagram of the video due diligence device provided in the embodiments of this application; Figure 10 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0018] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0020] Video due diligence refers to the process of verifying user identity and conducting background checks through remote video calls. In fintech scenarios, for example, video due diligence remotely verifies the authenticity of the borrower's declared office or business premises. In healthcare scenarios, it verifies the authenticity of the declared residential address of a medical insurance policyholder. Currently, the verification of user-declared premises in video due diligence relies entirely on manual review. Manual review depends on subjective human judgment, and inconsistent judgment standards lead to misjudgments and omissions, resulting in low accuracy.

[0021] Based on this, embodiments of this application provide a video due diligence method, a video due diligence device, an electronic device, and a computer-readable storage medium, aiming to improve the accuracy of video due diligence.

[0022] The video due diligence method, video due diligence device, electronic device, and computer-readable storage medium provided in this application are specifically described through the following embodiments. First, the video due diligence method in this application embodiment is described.

[0023] The video due diligence method provided in this application relates to the field of artificial intelligence technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the video due diligence method, but is not limited to the above forms.

[0024] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0025] The video-based due diligence method provided in this application can be applied to, for example... Figure 1In this application environment, the client communicates with the server via a network. The server can obtain ambient light and shadow images, audio signals, target geomagnetic intensity, and target illumination intensity of the video due diligence location through the client; extract image features from the ambient light and shadow images to obtain light and shadow vectors; extract audio features from the audio signals to obtain acoustic features; and perform consistency verification between the video due diligence location and the preset reporting location based on at least one of the light and shadow vectors, acoustic features, target geomagnetic intensity, and target illumination intensity to obtain video due diligence data; wherein, the video due diligence data is used to indicate whether the video due diligence location and the preset reporting location are consistent or inconsistent. In this application, automated video due diligence is achieved using at least one of the light and shadow vectors, acoustic features, target geomagnetic intensity, and target illumination intensity, eliminating reliance on human subjective judgment and improving the accuracy of video due diligence. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The following detailed description of specific embodiments further illustrates this application.

[0026] Figure 2 This is an optional flowchart of the video due diligence method provided in the embodiments of this application. Figure 2 The method may include, but is not limited to, steps S210 to S240.

[0027] Step S210: Acquire ambient light and shadow images, audio signals, target geomagnetic intensity, and target illumination intensity of the video due diligence location during the video due diligence process; Step S220: Extract image features from the ambient light and shadow image to obtain the light and shadow vector; Step S230: Extract audio features from the audio signal to obtain acoustic features; Step S240: Based on at least one of the light and shadow vector, acoustic features, target geomagnetic intensity, and target illumination intensity, perform consistency verification on the video due diligence location and the preset reporting location to obtain video due diligence data; wherein, the video due diligence data is used to indicate whether the video due diligence location and the preset reporting location are consistent or inconsistent.

[0028] Steps S210 to S270 as shown in the embodiments of this application utilize unified standards such as light and shadow vectors, acoustic features, target geomagnetic intensity, and target illumination intensity to achieve automated video due diligence, eliminating interference from human subjective judgment and improving the accuracy of video due diligence.

[0029] In step S210 of some embodiments, during the video due diligence process, ambient light and shadow images, audio signals, target geomagnetic intensity, and target illumination intensity of the video due diligence location are collected via a terminal device such as a mobile phone. The video due diligence location is the actual physical location of the investigated party, such as a borrower or policyholder, during a real-time video call. The ambient light and shadow image is an image with natural light and shadow, which can be an image of the environment outside a window or an image of an object with natural light and shadow. The audio signal can be direct sound from a sound source such as ambient white noise or user voice. If the video due diligence location is indoors, the audio signal will be reflected indoors; the audio signal can also be reflected sound, echo, reverberation, etc. The audio signal can be collected via a microphone. The target geomagnetic intensity is the magnetic induction intensity of the local magnetic field, which can be obtained using a magnetometer. The target illumination intensity is the visible light radiation energy received per unit area, used to indicate the ambient brightness. The target geomagnetic intensity and target illumination intensity are used to detect whether the video due diligence location conforms to the pre-set building structure benchmarks of the reporting location. The preset reporting location is the scene of the incident corresponding to the reporting service.

[0030] Please see Figure 3 In some embodiments, step S220 may include, but is not limited to, steps S310 to S330: Step S310: Perform semantic segmentation on the ambient light and shadow image to obtain the fixed object region; Step S320: Perform feature point detection on the fixed object region to obtain the key points at the root of the object and the key points at the top of the shadow. Step S330: Calculate the light and shadow vector based on the key points at the root of the object and the key points at the top of the shadow.

[0031] In step S310 of some embodiments, semantic segmentation is performed on the ambient lighting image to identify fixed objects perpendicular to the ground in the image, such as utility poles, window frames, and building edges, to obtain fixed object regions. Specifically, a semantic segmentation network is used to perform semantic segmentation on each frame of the continuous ambient lighting image to obtain the predicted category of each pixel in each frame of the ambient lighting image, and the frequency of occurrence of the predicted category of each pixel in each frame is counted. If the frequency of occurrence of the predicted category of the corresponding pixel in each frame is close to the frame number of the ambient lighting image, it means that the pixel is mostly predicted to be a window frame, utility pole, etc., and the pixel is determined to belong to the fixed object region. If the frequency of occurrence of the predicted category of the corresponding pixel in each frame differs significantly from the frame number of the ambient lighting image, it means that the pixel is a moving shadow, temporary clutter, etc., and the pixel is determined to belong to the dynamic region.

[0032] In step S320 of some embodiments, the contour edge of the fixed object region is extracted. The contour edge includes the bottom edge formed by the fixed object and its connecting plane. Corner detection is performed on the bottom edge to obtain candidate corner points. False corner points with jitter are removed from the candidate corner points to obtain the object root key points. A first normal and a second normal of the bottom edge are calculated. The directions of the first normal and the second normal are opposite, for example, the first normal is (A, B) and the second normal is (-A, -B). Along the first normal, starting from the object root key points, a first region is sampled in the fixed object region. Along the second normal, starting from the object root key points, a second region is sampled in the fixed object region. The areas of the first region and the second region are the same. The average brightness of the first region and the average brightness of the second region are calculated. If the average brightness of the first region is less than the average brightness of the second region, the first normal is determined to be the backlight direction; if the average brightness of the second region is less than the average brightness of the first region, the second normal is determined to be the backlight direction. Along the backlight direction, starting from the object root key points, a region of interest is extracted. Corner detection is performed on the region of interest to obtain the corner points of interest. If the number of points of interest (POIs) is greater than 1, calculate the distance between the object's root keypoint and the POIs, and select the POI with the largest distance as the shadow apex keypoint matching the object's root keypoint. If the number of POIs is equal to 1, then the POI is selected as the shadow apex keypoint. The shadow apex keypoint is the point where the shadow extends furthest.

[0033] In step S330 of some embodiments, a two-dimensional vector formed by pointing from the root keypoint of the object to the top keypoint of the shadow is used as the light and shadow vector. The light and shadow vector includes the light and shadow direction and the target light and shadow length. The root keypoint of the object has a first abscissa and a first ordinate, and the top keypoint of the shadow has a second abscissa and a second ordinate. A first difference between the second abscissa and the first abscissa is calculated, and a second difference between the second ordinate and the first ordinate is calculated. Based on the second difference and the first difference, the angle between the line connecting the root keypoint of the object and the top keypoint of the shadow and the horizontal coordinate axis is calculated using the arctangent function to obtain the light and shadow direction. The first difference is determined as the target light and shadow length.

[0034] Through the above steps S310 to S330, light and shadow vectors can be extracted from the ambient light and shadow image to achieve automated video due diligence based on the light and shadow vectors.

[0035] In step S230 of some embodiments, an acoustic blind deconvolution algorithm is used to analyze the reflection path of the audio signal at the video due diligence location, such as a wall or furniture, to obtain an impulse response curve. The time required for the audio signal to attenuate to 60 dB within the video due diligence location is measured to obtain the reverberation time. The impulse response curve and reverberation time are used as acoustic features to reflect the spatial volume and surface material of the video due diligence location.

[0036] Please see Figure 4 In some embodiments, step S240 may include, but is not limited to, steps S410 to S450: Step S410: Obtain the theoretical solar azimuth and theoretical solar altitude angle of the preset reporting location; Step S420: Calculate the first deviation between the theoretical solar azimuth and the direction of light and shadow; Step S430: Calculate the theoretical shadow length based on the theoretical solar altitude angle; Step S440: Calculate the second deviation between the theoretical shadow length and the target shadow length; Step S450: Based on at least one of the first deviation, second deviation, acoustic characteristics, target geomagnetic intensity, and target illumination intensity, perform consistency verification on the video due diligence location and the preset reporting location to obtain video due diligence data.

[0037] In step S410 of some embodiments, the theoretical solar azimuth and theoretical solar altitude angle of the preset reporting location are calculated. The theoretical solar azimuth is the clockwise deflection angle of the ray projected onto the horizontal plane by sunlight at the preset reporting location relative to the north reference. The theoretical solar altitude angle is the elevation angle between the incident sunlight ray and its projection onto the horizontal plane at the preset reporting location.

[0038] In step S420 of some embodiments, the absolute value of the difference between the theoretical solar azimuth angle and the direction of light and shadow is calculated to obtain the first deviation.

[0039] In step S430 of some embodiments, if the theoretical solar altitude angle is expressed as Let the object height be H, and the theoretical shadow length be L, then tan =H / L. The theoretical light and shadow length is the preset light and shadow length at the reporting location.

[0040] In step S440 of some embodiments, the absolute value of the difference between the theoretical light and shadow length and the target light and shadow length is calculated to obtain the second deviation.

[0041] In step S450 of some embodiments, the consistency of the video due diligence location and the preset reporting location is checked based on at least one of the first deviation, the second deviation, acoustic characteristics, target geomagnetic intensity and target illumination intensity, so as to determine whether the video due diligence location and the preset reporting location are consistent, and to obtain video due diligence data.

[0042] Through the above steps S410 to S450, automated video due diligence is performed based on multiple dimensions such as light and shadow vectors, acoustic features, target geomagnetic intensity, and target illumination intensity, thereby improving the accuracy of video due diligence.

[0043] Please see Figure 5 In some embodiments, step S410 may include, but is not limited to, steps S510 to S530: Step S510: Obtain the preset reporting time and the location coordinates of the preset reporting location; Step S520: Calculate the theoretical solar altitude angle based on the preset reporting time and location coordinates; Step S530: Calculate the theoretical solar azimuth angle based on the theoretical solar altitude angle, the preset reporting time, and the location coordinates.

[0044] In step S510 of some embodiments, the time of the instant the report is made is obtained to obtain the preset report time, and the location coordinates of the preset report location are obtained. The location coordinates are GPS coordinates, including longitude, latitude, etc.

[0045] In step S520 of some embodiments, the theoretical solar altitude angle and theoretical solar azimuth angle at the preset reporting location and preset reporting time are calculated. The theoretical solar altitude angle is the angle between the incident sunlight and the ground plane. The theoretical solar azimuth angle is the clockwise angle between the projection of the incident sunlight onto the ground plane and the local true north reference direction. The formula for calculating the theoretical solar altitude angle is expressed as: , in, Indicates the theoretical solar altitude angle; Indicates latitude; The solar declination angle can be calculated by setting a pre-defined reporting time. The hour angle is determined by the local time of the video due diligence location and is used to characterize the position of the sun relative to the meridian of the observation point.

[0046] In step S530 of some embodiments, the theoretical solar azimuth angle is calculated based on the theoretical solar altitude angle, the preset reporting time, and the location coordinates. The formula for calculating the theoretical solar azimuth angle is expressed as: , in, Indicates the theoretical solar altitude angle; Indicates latitude; The solar declination angle can be calculated by setting a pre-defined reporting time.

[0047] Through the above steps S510 to S530, the theoretical solar altitude angle and theoretical solar azimuth angle of the preset reporting location at the preset reporting time can be obtained, so as to perform geographical location verification of the consistency of light and shadow vectors based on the theoretical solar altitude angle and theoretical solar azimuth angle.

[0048] Please see Figure 6In some embodiments, step S450 may include, but is not limited to, steps S610 to S620: Step S610: Perform spatial semantic verification on the acoustic features to obtain semantic verification results; wherein, the semantic verification results are used to indicate whether the acoustic feature verification passes or fails. Step S620: Based on at least one of the first deviation, the second deviation, the semantic verification result, the target geomagnetic intensity, and the target illumination intensity, perform consistency verification on the video due diligence location and the preset reporting location to obtain video due diligence data.

[0049] In step S610 of some embodiments, different physical spaces (such as open workshops, compact office cubicles, and residential bedrooms) have distinctly different acoustic fingerprints. An acoustic fingerprint is a short, unique digital digest or feature code generated from acoustic features through filtering, hash encoding, and dimensionality reduction compression. The environmental acoustic fingerprint of the preset reporting location is obtained to obtain a standard acoustic fingerprint. Spatial semantic matching is performed between the acoustic features and the standard acoustic fingerprint. If they match, the semantic verification result is determined to be that the acoustic feature verification passed. If they do not match, the semantic verification result is determined to be that the acoustic feature verification failed.

[0050] The system can calculate the similarity between acoustic features and standard acoustic fingerprints. If the similarity is greater than a preset similarity threshold, the acoustic features and standard acoustic fingerprints are considered a match. If the similarity is less than or equal to the preset similarity threshold, the acoustic features and standard acoustic fingerprints are considered a mismatch. For example, if the preset reporting location is a large processing workshop, but the acoustic features indicate that the video due diligence location is a small, enclosed space, then the automatic warning indicates an environmental semantic conflict, meaning the acoustic features and standard acoustic fingerprints do not match.

[0051] In step S620 of some embodiments, a consistency check is performed on the video due diligence location and the preset reporting location based on at least one of the first deviation, the second deviation, the semantic verification result, the target geomagnetic intensity, and the target illumination intensity, in order to determine whether the two locations are consistent and to obtain video due diligence data.

[0052] Through steps S610 to S620, automated video due diligence is achieved by utilizing multiple dimensions such as the first deviation, the second deviation, the semantic verification result, the target geomagnetic intensity, and the target illumination intensity, thereby improving the accuracy of video due diligence.

[0053] Please see Figure 7 In some embodiments, step S620 may include, but is not limited to, steps S710 to S740: Step S710: Obtain the reference geomagnetic intensity and reference light intensity of the preset reporting location; Step S720: Compare the reference geomagnetic intensity and the target geomagnetic intensity to obtain a first comparison result; wherein, the first comparison result is used to indicate whether the reference geomagnetic intensity and the target geomagnetic intensity match or do not match; Step S730: Compare the reference light intensity and the target light intensity to obtain a second comparison result; wherein, the second comparison result is used to indicate whether the reference light intensity and the target light intensity match or do not match. Step S740: Based on at least one of the first deviation, the second deviation, the semantic verification result, the first comparison result, and the second comparison result, perform consistency verification on the video due diligence location and the preset reporting location to obtain video due diligence data.

[0054] In step S710 of some embodiments, a reference geomagnetic intensity and a reference light intensity at a preset reporting location are obtained. The reference geomagnetic intensity can exist in the form of a specific numerical value or in the form of a range.

[0055] In step S720 of some embodiments, a reference geomagnetic intensity and a target geomagnetic intensity are compared. If the difference between the reference and target geomagnetic intensities is small, or if the target geomagnetic intensity falls within the range of the reference geomagnetic intensity, then the first comparison result is determined to be a match between the reference and target geomagnetic intensities. If the difference between the reference and target geomagnetic intensities is large, or if the target geomagnetic intensity does not fall within the range of the reference geomagnetic intensity, then the first comparison result is determined to be a mismatch between the reference and target geomagnetic intensities.

[0056] In step S730 of some embodiments, the reference light intensity and the target light intensity are compared. If the difference between the reference light intensity and the target light intensity is small, or the target light intensity falls within the range of the reference light intensity, then the second comparison result is determined to be a match between the reference light intensity and the target light intensity. If the difference between the reference light intensity and the target light intensity is large, or the target light intensity does not fall within the range of the reference light intensity, then the second comparison result is determined to be a mismatch between the reference light intensity and the target light intensity.

[0057] In step S740 of some embodiments, the consistency of the video due diligence location and the preset reporting location is checked based on at least one of the first deviation, the second deviation, the semantic verification result, the first comparison result, and the second comparison result to obtain video due diligence data.

[0058] Through the above steps S710 to S740, automated video due diligence can be achieved.

[0059] Please see Figure 8 In some embodiments, step S740 may include, but is not limited to, step S810 or step S820: Step S810: If the first deviation is less than the first preset deviation threshold and the second deviation is less than the second preset deviation threshold, the semantic verification result indicates that the acoustic feature verification is passed, the first comparison result indicates that the reference geomagnetic intensity and the target geomagnetic intensity match, and the second comparison result indicates that the reference illumination intensity and the target illumination intensity match, then the video due diligence data is determined to be consistent with the video due diligence location and the preset reporting location. Step S820: If the first deviation is greater than or equal to the first preset deviation threshold, the second deviation is greater than or equal to the second preset deviation threshold, the semantic verification result indicates that the acoustic feature verification has failed, the first comparison result indicates that the reference geomagnetic intensity and the target geomagnetic intensity do not match, or the second comparison result indicates that the reference illumination intensity and the target illumination intensity do not match, then it is determined that the video due diligence data is inconsistent between the video due diligence location and the preset reporting location.

[0060] In step S810 of some embodiments, if the first deviation is less than the first preset deviation threshold, and the second deviation is less than the second preset deviation threshold, and the semantic verification result indicates that the acoustic feature verification is passed, and the first comparison result indicates that the reference geomagnetic intensity and the target geomagnetic intensity match, and the second comparison result indicates that the reference illumination intensity and the target illumination intensity match, then it is determined that the video due diligence data shows that the video due diligence location and the preset reporting location are consistent, and subsequent claims processing, credit granting, and other procedures can then be carried out. The first preset deviation threshold is a threshold preset for the detection deviation of the solar azimuth angle, and the second preset deviation threshold is a threshold preset for the detection deviation of the shadow length. The first preset deviation threshold and the second preset deviation threshold can be set according to the actual situation. For example, both the first preset deviation threshold and the second preset deviation threshold are 15°.

[0061] It should be noted that if the first deviation is less than the first preset deviation threshold and the second deviation is less than the second preset deviation threshold, it indicates that the light and shadow vector deviation between the video investigation location and the preset reporting location is small, and the video investigation location is a real site. If the first deviation is greater than or equal to the first preset deviation threshold, or the second deviation is greater than or equal to the second preset deviation threshold, it indicates that the light and shadow vector deviation between the video investigation location and the preset reporting location is large, and the video investigation location is a fake location, meaning that the user simulates the output of GPS coordinates through software, but cannot change the real sunlight and shadow of the shooting location.

[0062] In step S820 of some embodiments, if the first deviation is greater than or equal to the first preset deviation threshold, or the second deviation is greater than or equal to the second preset deviation threshold, or the semantic verification result indicates that the acoustic feature verification failed, or the first comparison result indicates that the reference geomagnetic intensity and the target geomagnetic intensity do not match, or the second comparison result indicates that the reference illumination intensity and the target illumination intensity do not match, then it is determined that the video due diligence data is inconsistent with the preset reporting location, and there is a risk of location forgery or staged shooting, which requires offline manual inspection or secondary verification.

[0063] The above steps S810 to S820 employ multi-dimensional video due diligence, enabling automated and accurate video due diligence.

[0064] Please see Figure 9 This application also provides a video due diligence device that can implement the above-described video due diligence method. The video due diligence device includes: The acquisition module 910 is used to acquire ambient light and shadow images, audio signals, target geomagnetic intensity, and target illumination intensity of the video due diligence location during the video due diligence process. The first feature extraction module 920 is used to extract image features from the ambient light and shadow image to obtain the light and shadow vector; The second feature extraction module 930 is used to extract audio features from the audio signal to obtain acoustic features; The verification module 940 is used to verify the consistency between the video due diligence location and the preset reporting location based on at least one of the light and shadow vector, audio signal, acoustic features, target geomagnetic intensity and target illumination intensity, and to obtain video due diligence data; wherein, the video due diligence data is used to indicate whether the video due diligence location and the preset reporting location are consistent or inconsistent.

[0065] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described video due diligence method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0066] Please see Figure 10 , Figure 10 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 1010 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 1020 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010 using the video due diligence method of the embodiments of this application. The input / output interface 1030 is used to implement information input and output; The communication interface 1040 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 1050 transmits information between various components of the device (e.g., processor 1010, memory 1020, input / output interface 1030, and communication interface 1040); The processor 1010, memory 1020, input / output interface 1030 and communication interface 1040 are connected to each other within the device via bus 1050.

[0067] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described video due diligence method.

[0068] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0069] The video due diligence method, video due diligence device, electronic device, and computer storage medium provided in this application embodiment utilize at least one of light and shadow vectors, acoustic features, target geomagnetic intensity, and target illumination intensity to achieve automated video due diligence, eliminating interference from human subjective judgment and improving the accuracy of video due diligence.

[0070] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0071] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0072] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0073] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0074] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0075] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0076] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0077] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0078] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0079] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0080] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with the knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0081] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A video due diligence method, characterized in that, The method includes: Acquire ambient light and shadow images, audio signals, target geomagnetic intensity, and target illumination intensity at the video due diligence location during the video due diligence process; Image features are extracted from the ambient light and shadow image to obtain the light and shadow vector; Audio features are extracted from the audio signal to obtain acoustic features; Based on at least one of the light and shadow vector, the acoustic features, the target geomagnetic intensity, and the target illumination intensity, a consistency check is performed on the video due diligence location and the preset reporting location to obtain video due diligence data; wherein, the video due diligence data is used to indicate whether the video due diligence location and the preset reporting location are consistent or inconsistent.

2. The method according to claim 1, characterized in that, The step of extracting image features from the ambient light and shadow image to obtain a light and shadow vector includes: Semantic segmentation is performed on the ambient light and shadow image to obtain fixed object regions; Feature point detection is performed on the fixed object region to obtain key points at the root of the object and key points at the top of the shadow. The light and shadow vector is calculated based on the key points at the root of the object and the key points at the top of the shadow.

3. The method according to claim 1, characterized in that, The light and shadow vector includes the light and shadow direction and the target light and shadow length. The consistency verification between the video due diligence location and the preset reporting location is performed based on at least one of the light and shadow vector, the acoustic features, the target geomagnetic intensity, and the target illumination intensity to obtain video due diligence data, including: Obtain the theoretical solar azimuth and theoretical solar altitude angle of the preset reporting location; Calculate the first deviation between the theoretical solar azimuth angle and the direction of light and shadow; Calculate the theoretical shadow length based on the theoretical solar altitude angle; Calculate the second deviation between the theoretical light and shadow length and the target light and shadow length; Based on at least one of the first deviation, the second deviation, the acoustic characteristics, the target geomagnetic intensity, and the target illumination intensity, the consistency of the video due diligence location and the preset reporting location is verified to obtain the video due diligence data.

4. The method according to claim 3, characterized in that, The step of obtaining the theoretical solar azimuth and theoretical solar altitude angle of the preset reporting location includes: Obtain the preset reporting time and the location coordinates of the preset reporting location; Calculate the theoretical solar altitude angle based on the preset reporting time and the location coordinates; The theoretical solar azimuth angle is calculated based on the theoretical solar altitude angle, the preset reporting time, and the location coordinates.

5. The method according to claim 3, characterized in that, The process of verifying the consistency between the video due diligence location and the preset reporting location based on at least one of the first deviation, the second deviation, the acoustic characteristics, the target geomagnetic intensity, and the target illumination intensity, to obtain the video due diligence data, includes: The acoustic features are subjected to spatial semantic verification to obtain a semantic verification result; wherein, the semantic verification result is used to indicate whether the acoustic feature verification passes or fails. Based on at least one of the first deviation, the second deviation, the semantic verification result, the target geomagnetic intensity, and the target illumination intensity, a consistency verification is performed on the video due diligence location and the preset reporting location to obtain the video due diligence data.

6. The method according to claim 5, characterized in that, The step of performing consistency verification on the video due diligence location and the preset reporting location based on at least one of the first deviation, the second deviation, the semantic verification result, the target geomagnetic intensity, and the target illumination intensity to obtain the video due diligence data includes: Obtain the reference geomagnetic intensity and reference light intensity of the preset reporting location; The reference geomagnetic intensity and the target geomagnetic intensity are compared to obtain a first comparison result; wherein, the first comparison result is used to indicate whether the reference geomagnetic intensity and the target geomagnetic intensity match or do not match; A second comparison result is obtained by comparing the reference light intensity and the target light intensity; wherein, the second comparison result is used to indicate whether the reference light intensity and the target light intensity match or do not match; Based on at least one of the first deviation, the second deviation, the semantic verification result, the first comparison result, and the second comparison result, a consistency verification is performed on the video due diligence location and the preset reporting location to obtain the video due diligence data.

7. The method according to claim 6, characterized in that, The step of performing consistency verification on the video due diligence location and the preset reporting location based on at least one of the first deviation, the second deviation, the semantic verification result, the first comparison result, and the second comparison result to obtain the video due diligence data includes: If the first deviation is less than the first preset deviation threshold, the second deviation is less than the second preset deviation threshold, the semantic verification result indicates that the acoustic feature verification is passed, the first comparison result indicates that the reference geomagnetic intensity and the target geomagnetic intensity match, and the second comparison result indicates that the reference illumination intensity and the target illumination intensity match, then it is determined that the video due diligence data is consistent with the video due diligence location and the preset reporting location; If the first deviation is greater than or equal to the first preset deviation threshold, the second deviation is greater than or equal to the second preset deviation threshold, the semantic verification result indicates that the acoustic feature verification has failed, the first comparison result indicates that the reference geomagnetic intensity and the target geomagnetic intensity do not match, or the second comparison result indicates that the reference illumination intensity and the target illumination intensity do not match, then it is determined that the video due diligence data is inconsistent between the video due diligence location and the preset reporting location.

8. A video due diligence device, characterized in that, The device includes: The acquisition module is used to acquire ambient light and shadow images, audio signals, target geomagnetic intensity, and target illumination intensity of the video due diligence location during the video due diligence process. The first feature extraction module is used to extract image features from the ambient light and shadow image to obtain a light and shadow vector; The second feature extraction module is used to extract audio features from the audio signal to obtain acoustic features; The verification module is used to verify the consistency between the video due diligence location and the preset reporting location based on at least one of the light and shadow vector, the audio signal, the acoustic features, the target geomagnetic intensity, and the target illumination intensity, to obtain video due diligence data; wherein, the video due diligence data is used to indicate whether the video due diligence location and the preset reporting location are consistent or inconsistent.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 7.

Images