Case voice explanation method, device, server and computer-readable storage medium

Abstract

The present application is applicable to the field of three-dimensional reconstruction technology, and provides a case voice explanation method, device, server and computer-readable storage medium, the method comprising: obtaining a three-dimensional model of the case scene; dividing the three-dimensional model of the case scene into case areas, and obtaining a three-dimensional case scene in the three-dimensional model of the case scene; wherein each three-dimensional case scene corresponds to a case area; associating a preset case explanation voice with the three-dimensional case scene; detecting the user's playback operation, determining the target three-dimensional case scene, and playing the target case explanation voice corresponding to the target three-dimensional case scene to the user. It can be seen that the present application can effectively improve the information presentation effect of the case explanation, so that users can understand the situation and details of the case scene more intuitively. By introducing the combination of three-dimensional models and voice explanations, users can experience a more immersive and interactive case explanation process, and improve the absorption and participation of information.

Description

Technical Field

[0001] The present application belongs to the field of three-dimensional reconstruction technology, and in particular, relates to a case voice explanation method, device, server and computer-readable storage medium. Background Art

[0002] In the process of handling cases such as criminal investigations and court trials, on-site investigation and explanation are crucial links. Traditional case explanations are mainly based on two-dimensional media such as text, pictures and videos, which cannot truly restore the spatial structure and details of the case scene. At the same time, the existing case explanation methods also have problems such as single information presentation and low user participation. Summary of the invention

[0003] The embodiments of the present application provide a case voice explanation method, device, server and computer-readable storage medium, which can solve the technical problems of the existing case explanation methods, such as single information presentation method and low user participation.

[0004] In a first aspect, an embodiment of the present application provides a case voice explanation method, characterized in that the method includes: Obtain a 3D model of the crime scene; Divide the three-dimensional model of the case scene into case areas to obtain three-dimensional case scenes in the three-dimensional model of the case scene; wherein each three-dimensional case scene corresponds to a case area; Associating a preset case explanation voice with the three-dimensional case scene; The user's playback operation is detected, a target three-dimensional case scene is determined, and a target case explanation voice corresponding to the target three-dimensional case scene is played to the user.

[0005] In a possible implementation of the first aspect, obtaining a three-dimensional model of the case scene includes: Obtaining a panoramic image to be processed; wherein the panoramic image to be processed refers to an image captured by a depth camera at the crime scene; generating a point cloud according to the panoramic image to be processed; The scene is reconstructed based on the point cloud to obtain a three-dimensional model of the case scene.

[0006] In a possible implementation of the first aspect, the three-dimensional model of the case scene is divided into regions to obtain a three-dimensional case scene in the three-dimensional model of the case scene, including: Identify the apartment area of ​​the three-dimensional model of the case scene; Add restricted areas within the described apartment area.

[0007] In a possible implementation of the first aspect, associating a preset case explanation voice with the three-dimensional case scene includes: In each of the three-dimensional case scenes, adding an identifier corresponding to the three-dimensional case scene; A data storage structure is constructed, wherein the data storage structure stores an identifier corresponding to each of the three-dimensional case scenes and a file path of the case explanation voice.

[0008] In a second aspect, an embodiment of the present application provides a case voice explanation device, characterized in that the device comprises: An acquisition module, used to acquire a three-dimensional model of the case scene; A region division module is used to divide the case scene three-dimensional model into case regions to obtain three-dimensional case scenes in the case scene three-dimensional model; wherein each three-dimensional case scene corresponds to a case region; An association module, used to associate a preset case explanation voice with the three-dimensional case scene; The detection module is used to detect the user's playback operation, determine the target three-dimensional case scene, and play the target case explanation voice corresponding to the target three-dimensional case scene to the user.

[0009] In a possible implementation manner of the second aspect, the acquisition module includes: An acquisition unit, used for acquiring a panoramic image to be processed; wherein the panoramic image to be processed refers to an image captured by a depth camera at the scene of the case; A generating unit, configured to generate a point cloud according to the panoramic image to be processed; A reconstruction unit is used to reconstruct the scene according to the point cloud to obtain a three-dimensional model of the case scene.

[0010] In a possible implementation manner of the second aspect, the area division module includes: An identification unit, used to identify the apartment area of ​​the three-dimensional model of the case scene; The first adding unit is used to add a restricted area in the apartment area.

[0011] In a possible implementation manner of the second aspect, the associating module includes: A second adding unit, used for adding an identifier corresponding to the three-dimensional case scene in each of the three-dimensional case scenes; A construction unit is used to construct a data storage structure, wherein the data storage structure stores an identifier corresponding to each of the three-dimensional case scenes and a file path of the case explanation voice.

[0012] In a third aspect, an embodiment of the present application provides a server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect above is implemented.

[0014] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: In the embodiment of the present application, a three-dimensional model of the case scene is obtained; the three-dimensional model of the case scene is divided into case areas to obtain a three-dimensional case scene in the three-dimensional model of the case scene; wherein each three-dimensional case scene corresponds to a case area; a preset case explanation voice is associated with the three-dimensional case scene; the user's playback operation is detected, the target three-dimensional case scene is determined, and the target case explanation voice corresponding to the target three-dimensional case scene is played to the user. It can be seen that the present application can effectively improve the information presentation effect of the case explanation, so that the user can understand the situation and details of the case scene more intuitively. By introducing the combination of three-dimensional models and voice explanations, users can experience a more immersive and interactive case explanation process, and improve the absorption and participation of information. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 is a schematic flow chart of the case area switching method provided by the present application; Figure 2 is a structural block diagram of a case area switching device provided in an embodiment of the present application; Figure 3 It is a schematic diagram of the structure of the server provided in the embodiment of the present application. Implementation

[0017] In the following description, specific details such as specific server structures and technologies are provided for illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known servers, devices, circuits, and methods are omitted to avoid obstructing the description of the present application with unnecessary details.

[0018] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0019] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0020] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0021] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0022] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0023] The technical solution of this application is described below.

[0024] Figure 1A schematic flow chart of a case area switching method provided by the present application is shown. As an example but not a limitation, the method is applied to a server, which is in communication connection with a user terminal. The method includes the following steps: Step S101, obtaining a three-dimensional model of the crime scene.

[0025] Step S201: Obtain the panoramic image to be processed The panoramic image to be processed refers to the image captured by the depth camera at the scene of the crime.

[0026] Step S202: generating a point cloud according to the panoramic image to be processed.

[0027] Exemplarily, generating a point cloud according to the panoramic image to be processed includes: Image preprocessing: Preprocess the captured panoramic images, such as removing lens distortion and noise reduction, to reduce interference and improve the accuracy of subsequent processing.

[0028] Feature extraction and matching: Using computer vision technology, feature points are detected in the panoramic image, and the features are described. Then, the feature matching algorithm is used to find the corresponding feature point pairs.

[0029] Camera calibration: Based on the matched feature point pairs, the camera's intrinsic calibration (such as camera focal length, principal point, radial distortion and other parameters) is calculated for subsequent 3D reconstruction.

[0030] Point cloud generation: Using the calculated camera parameters, the matched feature point pairs are triangulated, the 3D spatial coordinates corresponding to each feature point are calculated, and point cloud data is generated.

[0031] Step S203, reconstructing the scene based on the point cloud to obtain a three-dimensional model of the crime scene.

[0032] Exemplarily, scene reconstruction based on point cloud to obtain a three-dimensional model of the case scene includes: Point cloud registration: If the point cloud data is collected from multiple perspectives or time points, point cloud registration is required to align the point cloud data from different perspectives or time points. This can be achieved through algorithms such as feature matching and ICP (Iterative Closest Point).

[0033] Meshing: Meshing the registered point cloud data, converting the point cloud data into triangular facets or voxel meshes. You can use triangulation algorithms (such as Delaunay triangulation) or voxelization algorithms (such as Octree voxelization) for processing.

[0034] Hole filling and texture mapping: Based on the characteristics and needs of the scene, the meshed model is filled with holes to fill in possible missing parts, and texture mapping is performed to attach the texture information of the crime scene to the model to make it more realistic.

[0035] Optimization and detail processing: Detail processing and optimization of the scene reconstruction model can be performed as needed. Mesh simplification algorithms can be used to reduce the number of facets in the model and improve the performance and visualization of the model. At the same time, local detail reconstruction and high-precision model parts can also be performed.

[0036] Step S102, dividing the case scene three-dimensional model into case areas to obtain a three-dimensional case scene in the case scene three-dimensional model.

[0037] Among them, each three-dimensional case scene corresponds to a case area.

[0038] In a specific application, the three-dimensional model of the case scene is divided into regions to obtain a three-dimensional case scene in the three-dimensional model of the case scene, including: Step S301, identifying the apartment area of ​​the three-dimensional model of the crime scene, wherein the apartment area includes a plurality of room polygons and room vertices corresponding to the room polygons.

[0039] Exemplarily, a pre-trained RoomFormer floor plan recognition model is used to identify the floor plan areas.

[0040] Step S302, adding a restricted area within the apartment area.

[0041] It is understandable that prohibited or restricted-entry areas can be created in the three-dimensional model according to case investigation requirements to simulate on-site restrictions or protective measures when a case occurs.

[0042] Step S103, associating the preset case explanation voice with the three-dimensional case scene.

[0043] In specific applications, the preset case explanation voice is associated with the three-dimensional case scene, including: Step S401, adding an identifier corresponding to the three-dimensional case scene in each three-dimensional case scene; Step S402, construct a data storage structure, which stores an identifier corresponding to each three-dimensional case scene and a file path of the case explanation voice.

[0044] Exemplarily, the identifier is an identifier corresponding to a trigger point related to the three-dimensional case scene. The trigger point can be a clickable icon or button, or a visual element such as the boundary of the area.

[0045] Exemplarily, the data storage structure is: Case scenario information = { "Scene 1": "Explanation of audio file path 1", "Scene 2": "Explain the audio file path 2", "Scene 3": "Explanation audio file path 3".

[0046] Step S104, detecting the user's play operation, determining the target three-dimensional case scene, and playing the target case explanation voice corresponding to the target three-dimensional case scene to the user.

[0047] Exemplarily, after the trigger point is clicked, the explanation voice related to the trigger point is obtained through the logic in the trigger point event listener, and the corresponding playback operation is performed.

[0048] In the embodiment of the present application, a three-dimensional model of the case scene is obtained; the three-dimensional model of the case scene is divided into case areas to obtain a three-dimensional case scene in the three-dimensional model of the case scene; wherein each three-dimensional case scene corresponds to a case area; the preset case explanation voice is associated with the three-dimensional case scene; the user's playback operation is detected to determine the target three-dimensional case scene, and the target case explanation voice corresponding to the target three-dimensional case scene is played to the user. It can be seen that the present application can effectively improve the information presentation effect of the case explanation, so that users can more intuitively understand the situation and details of the case scene. By introducing the combination of three-dimensional models and voice explanations, users can experience a more immersive and interactive case explanation process, and improve the absorption and participation of information.

[0049] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. Corresponding to the case voice explanation method described in the above embodiment, Figure 2 A structural block diagram of a case voice explanation device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0050] Reference Figure 2 , the device comprises: An acquisition module 21 is used to acquire a three-dimensional model of the case scene; The area division module 22 is used to divide the case scene three-dimensional model into case areas to obtain three-dimensional case scenes in the case scene three-dimensional model; wherein each three-dimensional case scene corresponds to a case area; An association module 23, used to associate a preset case explanation voice with the three-dimensional case scene; The detection module 24 is used to detect the user's playback operation, determine the target three-dimensional case scene, and play the target case explanation voice corresponding to the target three-dimensional case scene to the user.

[0051] In a possible implementation, the acquisition module includes: An acquisition unit, used for acquiring a panoramic image to be processed; wherein the panoramic image to be processed refers to an image captured by a depth camera at the scene of the case; A generating unit, configured to generate a point cloud according to the panoramic image to be processed; A reconstruction unit is used to reconstruct the scene according to the point cloud to obtain a three-dimensional model of the case scene.

[0052] In a possible implementation, the region division module includes: An identification unit, used to identify the apartment area of ​​the three-dimensional model of the case scene; The first adding unit is used to add a restricted area in the apartment area.

[0053] In a possible implementation, the association module includes: A second adding unit, used for adding an identifier corresponding to the three-dimensional case scene in each of the three-dimensional case scenes; A construction unit is used to construct a data storage structure, wherein the data storage structure stores an identifier corresponding to each of the three-dimensional case scenes and a file path of the case explanation voice.

[0054] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0055] Figure 3 This is a schematic diagram of the structure of the server provided in the embodiment of the present application. Figure 3 As shown, the server 3 of this embodiment includes: at least one processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the at least one processor 30, and when the processor 30 executes the computer program 32, the steps in any of the above-mentioned method embodiments are implemented.

[0056] The server 3 may be a computing device such as a cloud server. The server may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will appreciate that Figure 3It is only an example of server 3 and does not constitute a limitation on server 3. Server 3 may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, etc.

[0057] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0058] In some embodiments, the memory 31 may be an internal storage unit of the server 3, such as a hard disk or memory of the server 3. In other embodiments, the memory 31 may also be an external storage device of the server 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the server 3. Further, the memory 31 may also include both an internal storage unit of the server 3 and an external storage device. The memory 31 is used to store operating servers, applications, boot loaders, data, and other programs, such as program codes of the computer programs. The memory 31 may also be used to temporarily store data that has been output or is to be output.

[0059] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0060] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0061] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0062] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0063] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another server, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0064] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0065] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A case voice explanation method, characterized in that: The method includes: Obtain a 3D model of the crime scene; Divide the three-dimensional model of the case scene into case areas to obtain three-dimensional case scenes in the three-dimensional model of the case scene; wherein each three-dimensional case scene corresponds to a case area; Associating a preset case explanation voice with the three-dimensional case scene; The user's playback operation is detected, a target three-dimensional case scene is determined, and a target case explanation voice corresponding to the target three-dimensional case scene is played to the user.

2. The case scene area switching method according to claim 1, characterized in that: Obtain a 3D model of the case scene, including: Obtaining a panoramic image to be processed; wherein the panoramic image to be processed refers to an image captured by a depth camera at the crime scene; generating a point cloud according to the panoramic image to be processed; The scene is reconstructed based on the point cloud to obtain a three-dimensional model of the case scene.

3. The case scene area switching method according to claim 1, characterized in that: The three-dimensional model of the case scene is divided into regions to obtain the three-dimensional case scene in the three-dimensional model of the case scene, including: Identify the apartment area of ​​the three-dimensional model of the case scene; Add restricted areas within the described apartment area.

4. The case scene area switching method according to any one of claims 1 to 3, characterized in that: Associating the preset case explanation voice with the three-dimensional case scene, including: In each of the three-dimensional case scenes, adding an identifier corresponding to the three-dimensional case scene; A data storage structure is constructed, wherein the data storage structure stores an identifier corresponding to each of the three-dimensional case scenes and a file path of the case explanation voice.

5. A case scene area switching method, characterized in that: The method includes: Obtain a 3D model of the crime scene; Divide the three-dimensional model of the case scene into case areas to obtain three-dimensional case scenes in the three-dimensional model of the case scene; wherein each three-dimensional case scene corresponds to a case area; Associating a preset case explanation voice with the three-dimensional case scene; The user's playback operation is detected, a target three-dimensional case scene is determined, and a target case explanation voice corresponding to the target three-dimensional case scene is played to the user.

6. A case voice explanation device, characterized in that: The method includes: An acquisition module, used to acquire a three-dimensional model of the case scene; A region division module is used to divide the case scene three-dimensional model into case regions to obtain three-dimensional case scenes in the case scene three-dimensional model; wherein each three-dimensional case scene corresponds to a case region; An association module, used to associate a preset case explanation voice with the three-dimensional case scene; The detection module is used to detect the user's playback operation, determine the target three-dimensional case scene, and play the target case explanation voice corresponding to the target three-dimensional case scene to the user.

7. The case voice explanation device according to claim 6, characterized in that: The acquisition module comprises: An acquisition unit, used for acquiring a panoramic image to be processed; wherein the panoramic image to be processed refers to an image captured by a depth camera at the scene of the case; A generating unit, configured to generate a point cloud according to the panoramic image to be processed; A reconstruction unit is used to reconstruct the scene according to the point cloud to obtain a three-dimensional model of the case scene.

8. The case voice explanation device according to claim 6, characterized in that: The area division module comprises: An identification unit, used to identify the apartment area of ​​the three-dimensional model of the case scene; The first adding unit is used to add a restricted area in the apartment area.

9. A server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images