Method, device, equipment and storage medium for constructing intelligent criminal investigation platform

Abstract

The present application discloses a construction method, device, equipment and storage medium of an intelligent criminal investigation platform, which relates to the field of intelligent criminal investigation technology, and discloses a construction method of an intelligent criminal investigation platform, including: obtaining a multimodal criminal investigation data set, and fine-tuning and training a generative large language model based on the multimodal criminal investigation data set to obtain a criminal investigation large model; constructing a criminal investigation knowledge base based on criminal investigation knowledge data through a text parsing strategy, and storing the criminal investigation knowledge base in a vector database to obtain a criminal investigation knowledge vector database; constructing a criminal investigation intelligent body based on the criminal investigation large model and the criminal investigation knowledge vector database in combination with a retrieval enhancement strategy, and the criminal investigation intelligent body is used to respond to the user's criminal investigation question and answer request; deploying the criminal investigation intelligent body to the business application layer to complete the construction of the intelligent criminal investigation platform. This solution optimizes the criminal investigation process by constructing an intelligent criminal investigation platform, thereby improving the efficiency of criminal investigation work.

Description

Technical Field

[0001] The present application relates to the field of intelligent criminal investigation technology, and in particular to a construction method, device, equipment and storage medium of an intelligent criminal investigation platform. Background Art

[0002] With the evolution of crime patterns and the advancement of technology, criminal investigation is facing unprecedented challenges, which requires police work to be more intelligent and efficient. Although data resources and big data investigation methods are constantly enriched, existing criminal investigation work relies on traditional and inefficient methods.

[0003] The deployment of existing traditional intelligent criminal investigation systems or intelligent platforms is relatively scattered, often relying on manual docking and conversion between multiple platforms. This not only increases the complexity of the system, but also leads to information blockage and low efficiency during use. How to build an intelligent criminal investigation platform to optimize the criminal investigation process and improve the efficiency of criminal investigation work has become a problem to be solved.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a method, device, equipment and storage medium for constructing an intelligent criminal investigation platform, aiming to solve the technical problem of how to construct an intelligent criminal investigation platform to optimize the criminal investigation process and improve the efficiency of criminal investigation work.

[0006] To achieve the above objectives, the present application proposes a method for constructing an intelligent criminal investigation platform, the method comprising:

[0007] Acquire a multimodal criminal investigation dataset, and fine-tune and train a generative large language model based on the multimodal criminal investigation dataset to obtain a criminal investigation large model;

[0008] Building a criminal investigation knowledge base based on criminal investigation knowledge data through a text parsing strategy, and storing the criminal investigation knowledge base in a vector database to obtain a criminal investigation knowledge vector database;

[0009] Based on the criminal investigation big model and the criminal investigation knowledge vector database, a criminal investigation intelligent agent is constructed in combination with a search enhancement strategy, and the criminal investigation intelligent agent is used to respond to the criminal investigation question and answer request of the user;

[0010] The criminal investigation agent is deployed to the business application layer to complete the construction of the intelligent criminal investigation platform.

[0011] In one embodiment, the step of constructing a criminal investigation knowledge base based on criminal investigation knowledge data by using a text parsing strategy, and storing the criminal investigation knowledge base in a vector database to obtain a criminal investigation knowledge vector database includes:

[0012] Preprocessing the criminal investigation knowledge data through a text parsing strategy to obtain a preprocessed text;

[0013] Performing entity recognition on the preprocessed text based on criminal investigation entity category information to obtain a criminal investigation word vector sequence;

[0014] Performing relationship extraction on the criminal investigation word vector sequence to obtain a criminal investigation relationship sequence;

[0015] Constructing a criminal investigation knowledge base by using the criminal investigation word vector sequence and the criminal investigation relationship sequence;

[0016] The criminal investigation knowledge base is converted into an embedded vector and stored in a vector database to obtain a criminal investigation knowledge vector database.

[0017] In one embodiment, the step of performing relationship extraction on the criminal investigation word vector sequence to obtain a criminal investigation relationship sequence includes:

[0018] Obtaining a position index sequence and a dimension index sequence of the criminal investigation word vector sequence;

[0019] Based on the position index sequence and the dimension index sequence, position information is integrated into the criminal investigation word vector sequence to obtain a first word vector sequence;

[0020] Calculate criminal investigation word pair features based on the first word vector sequence to obtain a first criminal investigation word pair feature matrix;

[0021] generating an entity category information matrix based on the criminal investigation entity category information;

[0022] The entity category information matrix is ​​concatenated with the first criminal investigation word pair feature matrix to obtain a second criminal investigation word pair feature matrix;

[0023] Calculating the horizontal axis attention and the vertical axis attention of the second criminal investigation word on the feature matrix to obtain an axial attention matrix;

[0024] Perform residual connection on the axial attention matrix and the second criminal investigation word pair feature matrix to obtain a criminal investigation relationship sequence.

[0025] In one embodiment, the step of fine-tuning and training the generative large language model based on the multimodal criminal investigation dataset to obtain the criminal investigation large model includes:

[0026] Loading a pre-trained generative large language model, and inputting the multimodal criminal investigation dataset into the generative large language model, wherein the generative large language model includes a multimodal encoder;

[0027] Encoding the data of the multimodal criminal investigation dataset by the multimodal encoder to obtain a criminal investigation image feature vector, a criminal investigation text feature vector and a criminal investigation label vector;

[0028] Calculating cosine similarity based on the criminal investigation image feature vector, the criminal investigation text feature vector, and the criminal investigation label vector;

[0029] The multimodal encoder is optimized according to the cosine similarity to minimize the similarity loss between the criminal investigation image feature vector and the criminal investigation text feature vector, thereby obtaining a criminal investigation large model.

[0030] In one embodiment, the step of calculating the cosine similarity based on the criminal investigation image feature vector, the criminal investigation text feature vector and the criminal investigation label vector includes:

[0031] assigning cross-modal attention weights to the criminal investigation image feature vector and the criminal investigation text feature vector based on the criminal investigation label vector;

[0032] weighting the criminal investigation image feature vector and the criminal investigation text feature vector based on the cross-modal attention weight to obtain a common feature vector;

[0033] Calculating the inner product of a preset hyperplane and the common feature vector to obtain a binary hash code sequence;

[0034] Determining a cross-modal hash code based on the hash value sequence;

[0035] The cosine similarity of the cross-modal hash codes is calculated.

[0036] In one embodiment, the step of constructing a criminal investigation agent based on the criminal investigation big model and the criminal investigation knowledge vector database in combination with a retrieval enhancement strategy includes:

[0037] Creating a criminal investigation workflow, wherein the criminal investigation workflow includes a pattern recognition workflow, a legal consultation workflow, and a report generation workflow;

[0038] Based on the retrieval enhancement strategy, the criminal investigation big model and the criminal investigation knowledge vector database are referenced in the criminal investigation workflow to obtain a target criminal investigation workflow;

[0039] A criminal investigation intelligent agent is constructed based on the target criminal investigation workflow, the preset human-computer interaction interface and the preset auxiliary plug-in.

[0040] In one embodiment, the step of obtaining a multimodal criminal investigation dataset includes:

[0041] Collect criminal investigation text data and criminal investigation image data;

[0042] Performing data cleaning on the criminal investigation text data and the criminal investigation image data to obtain pre-processed criminal investigation data;

[0043] The pre-processed criminal investigation data is annotated to obtain a multimodal criminal investigation data set.

[0044] In addition, to achieve the above-mentioned purpose, the present application also proposes a device for constructing an intelligent criminal investigation platform, and the device for constructing the intelligent criminal investigation platform includes:

[0045] A model fine-tuning module, used to obtain a multimodal criminal investigation dataset, and fine-tune the generative large language model based on the multimodal criminal investigation dataset to obtain a criminal investigation large model;

[0046] A knowledge construction module is used to construct a criminal investigation knowledge base based on criminal investigation knowledge data through a text analysis strategy, and store the criminal investigation knowledge base in a vector database to obtain a criminal investigation knowledge vector database;

[0047] An intelligent agent building module, used to build a criminal investigation intelligent agent based on the criminal investigation big model and the criminal investigation knowledge vector database in combination with a retrieval enhancement strategy, wherein the criminal investigation intelligent agent is used to respond to a criminal investigation question and answer request from a user;

[0048] The platform construction module is used to deploy the criminal investigation agent to the business application layer to complete the construction of the intelligent criminal investigation platform.

[0049] In addition, to achieve the above-mentioned purpose, the present application also proposes a device for constructing an intelligent criminal investigation platform, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the method for constructing an intelligent criminal investigation platform as described above.

[0050] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the method for constructing the intelligent criminal investigation platform as described above are implemented.

[0051] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the method for constructing an intelligent criminal investigation platform as described above.

[0052] One or more technical solutions proposed in this application have at least the following technical effects:

[0053] Acquire a multimodal criminal investigation data set, and fine-tune the generative large language model based on the multimodal criminal investigation data set to obtain a criminal investigation large model; construct a criminal investigation knowledge base based on the criminal investigation knowledge data through a text parsing strategy, and store the criminal investigation knowledge base in a vector database to obtain a criminal investigation knowledge vector database; construct a criminal investigation intelligent body based on the criminal investigation large model and the criminal investigation knowledge vector database in combination with a retrieval enhancement strategy, and the criminal investigation intelligent body is used to respond to the user's criminal investigation question and answer requests; deploy the criminal investigation intelligent body to the business application layer to complete the construction of an intelligent criminal investigation platform. The solution of this application integrates multimodal criminal investigation data sets (such as text, images) and fine-tunes the generative large language model, so that the criminal investigation large model can process and combine different types of criminal investigation data, providing more comprehensive and accurate support for criminal investigation analysis. By converting criminal investigation knowledge into a vector database and combining it with a text parsing strategy for criminal investigation knowledge, efficient retrieval based on semantic similarity can be achieved during retrieval to accurately recommend criminal investigation knowledge and cases, thereby improving the speed and accuracy of retrieval. By directly deploying criminal investigation agents to the business application layer and integrating them into the criminal investigation workflow, more efficient intelligent services can be achieved, providing instant intelligent decision-making support in case handling and responding to various query needs of criminal investigation personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0055] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0056] Figure 1 A flowchart of the first embodiment of the method for constructing an intelligent criminal investigation platform of the present application is provided;

[0057] Figure 2 A flowchart of a search enhancement strategy provided in Embodiment 1 of the method for constructing an intelligent criminal investigation platform of this application;

[0058] Figure 3 The intelligent criminal investigation platform construction architecture diagram provided in the first embodiment of the method for constructing the intelligent criminal investigation platform of this application;

[0059] Figure 4 A flowchart of the second embodiment of the method for constructing an intelligent criminal investigation platform of this application is provided;

[0060] Figure 5A schematic diagram of the module structure of a construction device of an intelligent criminal investigation platform according to an embodiment of the present application;

[0061] Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the method for constructing an intelligent criminal investigation platform in an embodiment of the present application.

[0062] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0063] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0064] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0065] The main solution of the embodiment of the present application is: obtaining a multimodal criminal investigation dataset, and fine-tuning and training a generative large language model based on the multimodal criminal investigation dataset to obtain a criminal investigation large model; constructing a criminal investigation knowledge base based on criminal investigation knowledge data through a text parsing strategy, and storing the criminal investigation knowledge base in a vector database to obtain a criminal investigation knowledge vector database; constructing a criminal investigation intelligent body based on the criminal investigation large model and the criminal investigation knowledge vector database in combination with a retrieval enhancement strategy, and the criminal investigation intelligent body is used to respond to the user's criminal investigation question and answer requests; and deploying the criminal investigation intelligent body to the business application layer to complete the construction of an intelligent criminal investigation platform.

[0066] In this embodiment, for ease of description, the following description is made by taking the identification intelligent platform system as the execution subject.

[0067] Since the deployment of existing traditional intelligent criminal investigation systems or intelligent platforms is relatively scattered, they often rely on manual docking and conversion between multiple platforms. This not only increases the complexity of the system, but also leads to information blockage and low efficiency during use. How to build an intelligent criminal investigation platform to optimize the criminal investigation process and improve the efficiency of criminal investigation work has become a problem to be solved.

[0068] This application provides a solution that integrates multimodal criminal investigation data sets (such as text, images) and fine-tunes the generative large language model, so that the criminal investigation large model can process and combine different types of criminal investigation data, providing more comprehensive and accurate support for criminal investigation analysis. By converting criminal investigation knowledge into a vector database and combining it with the text parsing strategy of criminal investigation knowledge, efficient retrieval based on semantic similarity can be achieved during retrieval to accurately recommend criminal investigation knowledge and cases, improve the speed and accuracy of retrieval, and directly deploy criminal investigation agents to the business application layer and integrate them into the criminal investigation workflow. More efficient intelligent services can be achieved, providing instant intelligent decision support in case handling, and responding to various query needs of criminal investigation personnel.

[0069] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, an intelligent platform system, etc. The following takes the intelligent platform system as an example to illustrate this embodiment and the following embodiments.

[0070] Based on this, the present application embodiment provides a method for constructing an intelligent criminal investigation platform, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the method for constructing an intelligent criminal investigation platform of the present application.

[0071] In this embodiment, the method for constructing the intelligent criminal investigation platform includes steps S10 to S40:

[0072] Step S10, obtaining a multimodal criminal investigation dataset, and fine-tuning and training a generative large language model based on the multimodal criminal investigation dataset to obtain a criminal investigation large model;

[0073] It should be understood that the multimodal criminal investigation dataset is a dataset obtained after preprocessing and annotating text data (such as case descriptions, testimonies, etc.) and criminal investigation image data (such as surveillance videos, crime scene photos, etc.). The generative large language model (LLM) is a natural language processing model based on deep learning, which can generate coherent text or answer questions based on user queries. Considering that the existing generative large language model learns the pattern of general language and cannot accurately extract the meaning and usage scenarios of professional terms in the specific field of the public security industry, that is, the field of criminal investigation, it is necessary to combine the data in the field of criminal investigation, that is, the multimodal criminal investigation dataset, to fine-tune the generative large language model and obtain a large model of the public security industry (i.e., the criminal investigation large model). The criminal investigation large model obtained by fine-tuning the training by integrating the multimodal criminal investigation dataset can learn more accurate criminal investigation knowledge and logic, better understand and process professional terms in the field of criminal investigation, and improve the accuracy in criminal investigation tasks.

[0074] In a feasible implementation, the step of obtaining a multimodal criminal investigation dataset in step S10 may include steps S101 to S103:

[0075] Step S101, collecting criminal investigation text data and criminal investigation image data;

[0076] It should be noted that criminal investigation text data and criminal investigation image data can be collected through the data collection system. Criminal investigation text data refers to various text information related to criminal investigation, including but not limited to case reports, testimonies, investigation records, investigation reports, case summaries, forensic identification reports, on-site investigation records, etc., which can provide criminal investigators with detailed information and clues of the case. Criminal investigation image data refers to image or video information collected during the criminal investigation process, including but not limited to crime scene photos, surveillance videos, evidence photos, forensic scene images, DNA detection images, fingerprint images, etc., which can provide intuitive visual evidence for criminal investigation analysis and help investigators analyze the crime scene, the appearance characteristics of suspects, the arrangement of evidence, etc.

[0077] Step S102, performing data cleaning on the criminal investigation text data and the criminal investigation image data to obtain pre-processed criminal investigation data;

[0078] It should be noted that preprocessing criminal investigation data includes preprocessing text data and preprocessing image data.

[0079] For the collected criminal investigation text data, in the data cleaning process, it is necessary to remove irrelevant information in the text (such as garbled characters, meaningless characters, advertisements, templates, etc.), correct errors in the text (such as typos, punctuation errors, incomplete sentence structure, etc.), unify the format of time and place information in the text (such as date format, standardization of place nouns), and remove duplicate text content to obtain pre-processed text data. For the collected criminal investigation image data, in the data cleaning process, it is necessary to first screen it, remove images with too low resolution or blur, and then crop and adjust the images to unify the image size and resolution, ensure image clarity and quality, and obtain pre-processed image data.

[0080] Step S103, annotating the pre-processed criminal investigation data to obtain a multimodal criminal investigation data set.

[0081] It should be noted that after obtaining the pre-processed criminal investigation data, the pre-processed criminal investigation data can be annotated by the data annotation system to obtain a multimodal criminal investigation data set. Specifically, when annotating the pre-processed text data and the pre-processed image data, the case type, target person, and case evidence corresponding to the pre-processed text data and the pre-processed image data can be annotated. Among them, the case type can be the corresponding criminal case type; the target person can be the suspect, victim, witness, etc. involved in the case; the case evidence can be fingerprints, DNA evidence, physical evidence, etc.

[0082] In a feasible implementation, the step of fine-tuning and training the generative large language model based on the multimodal criminal investigation dataset to obtain the criminal investigation large model in step S10 may include steps S11 to S14:

[0083] Step S11, loading a pre-trained generative large language model, and inputting the multimodal criminal investigation dataset into the generative large language model, wherein the generative large language model includes a multimodal encoder;

[0084] It should be understood that the pre-trained generative large language model is a language model trained with a large amount of unlabeled data, which can learn from massive data how to generate coherent text, understand contextual meaning, and generate reasonable output in different contexts. The generative large language model includes an encoder and a decoder, wherein the encoder is a multimodal encoder, which can encode data of each modality separately through different sub-networks, and is used to encode the input data and generate a query vector. The decoder is used to generate the corresponding question-answering result according to the query vector.

[0085] Step S12, encoding the data of the multimodal criminal investigation dataset by the multimodal encoder to obtain a criminal investigation image feature vector, a criminal investigation text feature vector and a criminal investigation label vector;

[0086] It should be noted that the multimodal encoder includes an image encoder, a text encoder, and a label encoder. The image encoder is used to obtain image data in the multimodal criminal investigation dataset, extract features from the image data through a convolutional neural network or other image processing network, and obtain a criminal investigation image feature vector V image The text encoder is used to obtain text data in the multimodal criminal investigation dataset, extract features from the text data through a recurrent neural network or other text processing network, and obtain the criminal investigation text feature vector V text The label encoder is used to obtain the labeled data (such as case type, target person, and case evidence) in the multimodal criminal investigation dataset, and maps each category to a feature vector through one-hot encoding or label encoding to obtain the criminal investigation label vector V label .

[0087] Step S13, calculating cosine similarity according to the criminal investigation image feature vector, the criminal investigation text feature vector and the criminal investigation label vector;

[0088] It should be understood that the cosine similarity between the criminal investigation image feature vector and the criminal investigation text feature vector can be calculated through the criminal investigation label vector to measure the degree of matching between the image and the text, ensuring that the image data and the text data jointly describe the same case scene.

[0089] In a feasible implementation, step S13 may include: assigning cross-modal attention weights to the criminal investigation image feature vector and the criminal investigation text feature vector based on the criminal investigation label vector; weighting the criminal investigation image feature vector and the criminal investigation text feature vector based on the cross-modal attention weights to obtain a common feature vector; calculating the inner product of a preset hyperplane and the common feature vector to obtain a binary hash code sequence; determining a cross-modal hash code based on the hash value sequence; and calculating the cosine similarity of the cross-modal hash code.

[0090] It should be noted that according to the criminal investigation label vector V label Can be the criminal investigation image feature vector V image and the criminal investigation text feature vector V text Assign different attention weights. Specifically, the inner product of the criminal investigation label vector and the image feature vector, and the inner product of the criminal investigation label vector and the text feature vector can be calculated to obtain the correlation representation. The correlation representation is converted into a weight through a normalization function so that the sum of all attention weights is 1. The cross-modal attention weight includes the image feature attention weight A image and text feature attention weight A text , the cross-modal attention weight calculation formula of the criminal investigation label vector to the criminal investigation image feature vector and the criminal investigation text feature vector is as follows:

[0091]

[0092]

[0093] in, represents the inner product of the criminal investigation label vector and the criminal investigation image feature vector, It represents the inner product of the criminal investigation label vector and the criminal investigation text feature vector. Softmax is the standard normalization function.

[0094] Attention weight A according to image features image and text feature attention weight A text , respectively for the criminal investigation image feature vector V image and the criminal investigation text feature vector V textBy weighting, the first common feature vector and the second common feature vector can be obtained, wherein the first common feature vector is the feature vector V of the criminal investigation image. image The second common feature vector is the feature vector V of the criminal investigation text. text The public feature vector includes a first public feature vector and a second public feature vector.

[0095] In addition, it should be noted that the preset hyperplane is a set of random vectors r1, r2, ..., r generated randomly. k The hyperplane defined is used to calculate the projection of the feature vector in the hash space, where k is the dimension of the hash (i.e., the hash length). For each hyperplane r i , calculate its inner product with the common feature vector, map the inner product to 1 (if the inner product is positive) or -1 (if the inner product is negative) through the sign function, and get the hash values ​​h1, h2, ..., h k , which is a binary hash code sequence. All hash values ​​h1, h2, ..., h k concatenated into a binary vector to obtain the cross-modal hash code H. The cross-modal hash code H includes the image hash code H I and the text hash code H T After obtaining the cross-modal hash codes of the criminal investigation image feature vector and the criminal investigation text feature vector, the cosine similarity between these codes can be calculated to measure the similarity between the two vectors in the hash space. The image hash code H I and the text hash code H T The cosine similarity of The calculation formula is as follows:

[0096]

[0097] Step S14, optimizing the multimodal encoder according to the cosine similarity to minimize the similarity loss between the criminal investigation image feature vector and the criminal investigation text feature vector, and obtaining a criminal investigation large model.

[0098] It should be understood that in the fine-tuning training stage of the generative language large model, after obtaining the cosine similarity of the criminal investigation image feature vector and the criminal investigation text feature vector, the loss function can be calculated based on the cosine similarity, the gradient of the loss function can be calculated by the back propagation algorithm, the weights of the image encoder and the text encoder in the multimodal encoder are updated, and the training is continuously iterated until the loss converges to minimize the similarity loss of the criminal investigation image feature vector and the criminal investigation text feature vector, thereby obtaining the criminal investigation large model.

[0099] Specifically, for the paired criminal investigation image feature vectors and criminal investigation text feature vectors extracted from the modal criminal investigation dataset, the contrast loss can be used to optimize the multimodal encoder to encourage the multimodal encoder to maximize the similarity on positive pairs (i.e., images and texts with criminal investigation semantics related) and minimize the similarity on negative pairs (i.e., images and texts with criminal investigation semantics unrelated). Contrastive loss function L constractive The calculation formula is as follows:

[0100]

[0101] Where y is the label (1 for positive pairs and 0 for negative pairs), and m is a set hyperparameter (usually a small positive number such as 0.2) indicating the minimum distance difference between positive and negative pairs.

[0102] Step S20, constructing a criminal investigation knowledge base based on the criminal investigation knowledge data through a text parsing strategy, and storing the criminal investigation knowledge base in a vector database to obtain a criminal investigation knowledge vector database;

[0103] It should be understood that criminal investigation knowledge data includes criminal investigation case data, legal text data, investigation report data and other public security industry knowledge data in text form, which can be obtained through public security organs, courts, public documents, social media and other channels. The text parsing strategy is used to convert criminal investigation knowledge data into structured criminal investigation knowledge that can be effectively stored and processed to obtain a criminal investigation knowledge base. Specifically, the text parsing strategy can be used to perform entity recognition and relationship extraction on criminal investigation knowledge data to obtain a criminal investigation knowledge base in the form of a knowledge graph. For efficient storage and retrieval, a vector database (such as FAISS, Pinecone, etc.) can be used to store and index the criminal investigation knowledge base to obtain a criminal investigation knowledge vector database.

[0104] In a feasible implementation, step S20 may include steps S21 to S25:

[0105] Step S21, preprocessing the criminal investigation knowledge data through a text parsing strategy to obtain a preprocessed text;

[0106] It should be understood that after obtaining criminal investigation knowledge data in text form, it is necessary to preprocess it through a text parsing strategy to obtain a structured preprocessed text. Specifically, in the process of preprocessing the text, it is necessary to first clean the text, remove irrelevant symbols, numbers, stop words, etc., and then segment the cleaned text and perform part-of-speech tagging to obtain the preprocessed text.

[0107] Step S22, performing entity recognition on the preprocessed text based on the criminal investigation entity category information to obtain a criminal investigation word vector sequence;

[0108] It should be understood that entities are specific objects or concepts that can be identified and classified in the preprocessed text. In the field of criminal investigation, entities are usually related to key information related to the case. Criminal investigation entity category information is predefined entity category information in the field of criminal investigation, which is used to define criminal investigation entities. Specifically, criminal investigation entities include but are not limited to entities of persons involved in the case (suspects, victims, witnesses), case time entities, case location entities, and case event entities. The goal of entity recognition is to identify criminal investigation entities in the text and mark them as predefined categories. The criminal investigation word vector sequence is a word vector sequence obtained by converting criminal investigation entities in the text into vector representations through entity recognition. Named Entity Recognition (NER) technology can be used to extract criminal investigation entities in the preprocessed text, convert criminal investigation entities into word vectors, and obtain a criminal investigation word vector sequence.

[0109] Specifically, the preprocessed text is input into the pretrained language model, which will segment and tokenize the input preprocessed text and convert it into a sequence of words or subwords. The sentence boundaries are marked by segmentation markers to obtain the criminal investigation sentence segmentation results. Based on the criminal investigation sentence segmentation results, a criminal investigation word sentence sequence can be obtained. The criminal investigation sentences in the criminal investigation word sentence sequence are input into the pretrained language model. The pretrained language model generates a context-related embedding vector for each criminal investigation entity based on context information through a multi-layer Transformer architecture to obtain a criminal investigation word vector sequence. Assuming that the number of entities in the input criminal investigation sentence is n, the output of the pre-trained language model is a word vector matrix of shape [n, d], that is, a criminal investigation word vector sequence, where d is the dimension of the word vector.

[0110] Step S23, performing relationship extraction on the criminal investigation word vector sequence to obtain a criminal investigation relationship sequence;

[0111] It should be noted that relation extraction aims to identify the semantic relations between different criminal investigation entities from the preprocessed text. The criminal investigation relation sequence contains the relation information between criminal investigation entities in the text, which is an n×n relation matrix, where n is the number of entities.

[0112] In a feasible implementation, step S23 may include steps A11 to A17:

[0113] Step A11, obtaining a position index sequence and a dimension index sequence of the criminal investigation word vector sequence;

[0114] It should be noted that for the criminal investigation word vector sequence with shape [n, d] , according to the number of criminal investigation word vectors in the criminal investigation word vector sequence, a position index sequence can be generated According to the criminal investigation word vector dimension of the criminal investigation word vector sequence, a dimension index sequence can be generated , d is the dimension of the criminal investigation word vector.

[0115] Step A12, integrating position information into the criminal investigation word vector sequence based on the position index sequence and the dimension index sequence to obtain a first word vector sequence;

[0116] It should be noted that for each dimension j of the criminal investigation word vector at each position i in the criminal investigation word vector sequence, the sine and cosine values ​​are calculated to obtain the position embedding vector E ij , position embedding vector E ij The calculation formula is as follows:

[0117]

[0118] Among them, E ij represents the embedding value of the i-th position and the j-th dimension, and d is the dimension of the criminal investigation word vector. ij The position vector matrix E can be obtained. After concatenating the criminal investigation word vector sequence L with the position vector matrix E, the first word vector sequence L1 incorporating the position information can be obtained.

[0119] Step A13, calculating criminal investigation word pair features based on the first word vector sequence to obtain a first criminal investigation word pair feature matrix;

[0120] It should be noted that the criminal investigation word pair feature is a feature extracted from the text in the form of word pairs, and the word pairs represent the relationship between criminal investigation entities. For the first word vector sequence L1, the criminal investigation word pair feature can be calculated by the following relationship extraction formula: :

[0121]

[0122] in, and are the i-th criminal investigation word vector and the j-th criminal investigation word vector in the first word vector sequence L1, respectively. W is a learnable weight matrix. b1 is a bias term with a dimension of d×d, where d is the dimension of the criminal investigation word vector. is an activation function, such as ReLU or sigmoid, For criminal investigation word vector and The criminal investigation word pair features obtained by calculation can obtain the first criminal investigation word pair feature matrix M after calculating each pair of word vectors in the first word vector sequence, and the dimension of M is n×n×d.

[0123] Step A14, generating an entity category information matrix based on the criminal investigation entity category information;

[0124] It should be noted that by encoding the criminal investigation entity category in the criminal investigation entity category information, a criminal investigation entity category vector can be obtained. The criminal investigation entity category vector is input into the pre-trained language model to obtain a category word vector matrix. For a criminal investigation entity category vector with an input length of m, the category word vector sequence output by the pre-trained language model is , a word vector matrix of shape [m, d], that is, the criminal investigation word vector sequence, where m is the number of criminal investigation entity categories in the criminal investigation entity category information, and d is the dimension of the word vector. Get the criminal investigation word vector sequence Then, according to the category word vector sequence U and the criminal investigation word vector sequence L, a vector Z representing the category information of the criminal investigation entity can be constructed. ij , and then we get the entity category information matrix Z, the dimension of Z is n×n×d.

[0125] Step A15, concatenating the entity category information matrix with the first criminal investigation word pair feature matrix to obtain a second criminal investigation word pair feature matrix;

[0126] It should be understood that by splicing the entity category information matrix Z with the first criminal investigation word pair feature matrix M, a second criminal investigation word pair feature matrix W incorporating criminal investigation entity category information can be obtained. Specifically, since the above-mentioned entity category information matrix Z and the first criminal investigation word pair feature matrix M have the same dimension, the two matrices can be directly spliced ​​together along the specified dimension (along the column or along the row) to obtain the second criminal investigation word pair feature matrix.

[0127] Step A16, calculating the horizontal axis attention and the vertical axis attention of the second criminal investigation word on the feature matrix to obtain an axial attention matrix;

[0128] It should be noted that after obtaining the second criminal investigation word pair feature matrix W, the second criminal investigation word pair feature matrix W is obtained through the weight matrix at each position After the linear projection on , the horizontal and vertical attention of the second criminal investigation word on the feature matrix can be calculated based on the linear projection. Specifically, the horizontal attention and vertical attention The calculation formula is as follows:

[0129]

[0130]

[0131] Among them, q represents query, k represents key, and v represents value. q, k, and v are all linear projections of the feature matrix W of the second criminal investigation word at position (s, o). The softmax function is used to convert the dot product of the query and the key into a probability distribution to represent the importance of different positions. is the eigenvector of the second criminal investigation word pair feature matrix W at position (s, o), and n is the number of rows and columns of the second criminal investigation word pair feature matrix. After integration, we can get the axial attention matrix.

[0132] Step A17, performing residual connection on the axial attention matrix and the second criminal investigation word pair feature matrix to obtain a criminal investigation relationship sequence.

[0133] It should be understood that after obtaining the axial attention matrix, the corresponding elements of the second criminal investigation word pair feature matrix and the axial attention matrix are added (i.e., residual connection is performed) to obtain a criminal investigation relationship sequence, which is a criminal investigation relationship matrix that introduces category information and integrates the characteristics of the axial attention.

[0134] Step S24, constructing a criminal investigation knowledge base by combining the criminal investigation word vector sequence and the criminal investigation relationship sequence;

[0135] It should be understood that after obtaining the criminal investigation word vector sequence and the criminal investigation relationship sequence, a criminal investigation knowledge graph can be constructed based on the criminal investigation word vector sequence and the criminal investigation relationship sequence to obtain a criminal investigation knowledge base. The criminal investigation knowledge graph uses the criminal investigation entities in the criminal investigation word vector sequence as nodes of the knowledge graph and the criminal investigation relationship sequence in the criminal investigation relationship sequence as edges of the knowledge graph.

[0136] Step S25, converting the criminal investigation knowledge base into an embedded vector and storing it in a vector database to obtain a criminal investigation knowledge vector database.

[0137] It should be noted that after obtaining the criminal investigation knowledge base, each entity and relationship in the criminal investigation knowledge base can be mapped to a vector space through a suitable embedding model such as TransE, DistMult, ComplEx, etc., to obtain the corresponding embedding vector, and the embedded vector is stored in the vector database to obtain the criminal investigation knowledge vector database.

[0138] Step S30, constructing a criminal investigation intelligent agent based on the criminal investigation big model and the criminal investigation knowledge vector database in combination with a search enhancement strategy, wherein the criminal investigation intelligent agent is used to respond to the criminal investigation question and answer request of the user;

[0139] It should be noted that the retrieval enhancement strategy can be implemented through the RAG retrieval enhancement system. Through the retrieval enhancement strategy, the information retrieval technology can be combined with the generation model to enhance the model's answering ability based on external knowledge bases such as the criminal investigation knowledge vector database. The criminal investigation agent is an intelligent system that aims to provide intelligent services based on the knowledge and information in the field of criminal investigation. It can be built through the agent construction tool.

[0140] For example, refer to Figure 2 , Figure 2This is the search enhancement strategy process provided in the first embodiment of the method for constructing the intelligent criminal investigation platform of this application. Figure 2 As shown in the figure, when a user makes a criminal investigation question and answer request through the user interaction interface, the criminal investigation intelligent body will input the text data and image data in the criminal investigation question and answer request into the multimodal encoder of the public security industry big model (i.e., the criminal investigation big model) to obtain a query vector, retrieve relevant information from the criminal investigation knowledge vector database based on the query vector, and pass the retrieved content as context information to the decoder of the public security industry big model to generate a criminal investigation question and answer result.

[0141] Step S40, deploying the criminal investigation agent to the business application layer to complete the construction of the intelligent criminal investigation platform.

[0142] It should be understood that after the criminal investigation intelligent body is constructed through the intelligent body construction tool, the criminal investigation intelligent body can be connected with various business systems, such as case management systems, criminal investigation analysis platforms, intelligent query platforms, etc., and the criminal investigation intelligent body can be deployed to the business application layer to complete the construction of the intelligent criminal investigation platform.

[0143] For example, Figure 3 This is a diagram of the intelligent criminal investigation platform construction architecture provided in the first embodiment of the construction method of the intelligent criminal investigation platform of this application. Figure 3 As shown in the figure, the construction process of the intelligent criminal investigation platform is based on a layered architecture, including a model capability layer and a business application layer. Among them, the model capability layer specifically includes the model management layer, the model capability layer, and the agent development layer from the bottom to the top. In the model management layer, criminal investigation text data and criminal investigation image data can be collected through the data collection system, and the criminal investigation text data and criminal investigation image data can be annotated through the data annotation system to obtain a multimodal criminal investigation data set. After receiving the incoming multimodal criminal investigation data set through the large model training platform, the pre-trained generative large language model will be loaded, and the generative large language model will be fine-tuned and trained based on the multimodal criminal investigation data set to obtain a public security industry large model (i.e., a criminal investigation large model). In the model enhancement layer, the DOC parsing model of the knowledge base management system will be used to process the criminal investigation knowledge data based on the text parsing strategy to obtain a criminal investigation knowledge base, and the criminal investigation knowledge base will be converted into an embedded vector through the embedding model and stored in a high-performance vector database to obtain a criminal investigation knowledge vector database. The core logic of the criminal investigation intelligent agent is created through the RAG retrieval enhancement system based on the public security industry large model and the criminal investigation knowledge vector database, combined with the retrieval enhancement strategy. At the agent development layer, we develop an agent intelligent agent platform that conforms to the core logic through the low-code agent development platform, and build a criminal investigation intelligent agent with AI question-answering assistant functions. The criminal investigation intelligent agent is deployed to the business application layer to complete the construction of the intelligent criminal investigation platform.

[0144] This embodiment provides a method for constructing an intelligent criminal investigation platform, which obtains a multimodal criminal investigation data set, and fine-tunes and trains a generative large language model based on the multimodal criminal investigation data set to obtain a criminal investigation large model; constructs a criminal investigation knowledge base based on criminal investigation knowledge data through a text parsing strategy, and stores the criminal investigation knowledge base in a vector database to obtain a criminal investigation knowledge vector database; constructs a criminal investigation intelligent body based on the criminal investigation large model and the criminal investigation knowledge vector database in combination with a retrieval enhancement strategy, and the criminal investigation intelligent body is used to respond to the user's criminal investigation question and answer request; and deploys the criminal investigation intelligent body to the business application layer to complete the construction of the intelligent criminal investigation platform. The solution of this application fine-tunes and trains a generative large language model by integrating a multimodal criminal investigation data set, so that the criminal investigation large model can process and combine different types of criminal investigation data, providing more comprehensive and accurate support for criminal investigation analysis. By converting criminal investigation knowledge into a vector database and combining it with a text parsing strategy for criminal investigation knowledge, efficient retrieval based on semantic similarity can be achieved during retrieval to accurately recommend criminal investigation knowledge and cases, thereby improving the speed and accuracy of retrieval. By directly deploying criminal investigation agents to the business application layer and integrating them into the criminal investigation workflow, more efficient intelligent services can be achieved, providing instant intelligent decision-making support in case handling and responding to various query needs of criminal investigation personnel.

[0145] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 4 , step S30 may include steps S31 to S33:

[0146] Step S31, creating a criminal investigation workflow, wherein the criminal investigation workflow includes a pattern recognition workflow, a legal consultation workflow, and a report generation workflow;

[0147] It should be noted that the criminal investigation workflow is a series of steps and tasks involved in the criminal investigation process. The criminal investigation workflow includes at least pattern recognition workflow, legal consultation workflow and report generation workflow. Among them, the pattern recognition workflow is used to identify crime patterns, such as modus operandi, time, location, etc., to predict crime trends or identify serial crimes; the legal consultation workflow is used to provide rapid retrieval of legal provisions, precedents and legal interpretations to provide legal support for criminal investigators; the report generation workflow is used to automatically generate case reports, including case summaries, evidence lists, analysis results, etc., to simplify document work.

[0148] Step S32, referencing the criminal investigation big model and the criminal investigation knowledge vector database in the criminal investigation workflow based on the retrieval enhancement strategy to obtain a target criminal investigation workflow;

[0149] It should be noted that the criminal investigation big model includes an encoder and a decoder, wherein the encoder is a multimodal encoder obtained through fine-tuning and training of a multimodal criminal investigation data set. The criminal investigation big model and the criminal investigation knowledge vector database are referenced in the criminal investigation workflow, that is, the retrieval question-and-answer function is integrated into the criminal investigation workflow through the RAG retrieval enhancement system. The retrieval question-and-answer function encodes the user's criminal investigation question-and-answer request through the multimodal encoder of the criminal investigation big model to obtain a query vector, and accesses the criminal investigation knowledge vector stored in the vector database from the criminal investigation knowledge vector database based on the query vector, and inputs it into the decoder of the criminal investigation big model to obtain the criminal investigation question-and-answer result. The target criminal investigation workflow is the criminal investigation workflow that references the criminal investigation big model and the criminal investigation knowledge vector database.

[0150] Step S33, constructing a criminal investigation intelligent agent based on the target criminal investigation workflow, the preset human-computer interaction interface and the preset auxiliary plug-in.

[0151] It should be noted that the target criminal investigation workflow defines the core logic of the agent. The preset human-computer interaction interface is a designed user interface for criminal investigators to interact with the agent in language. The preset auxiliary plug-in is a development auxiliary tool or plug-in used to enhance the function of the agent. Specifically, the preset auxiliary plug-in can be a speech recognition plug-in, a document extraction plug-in, and a data visualization plug-in. Among them, the speech recognition plug-in is used to convert audio or voice recordings (such as suspect confessions, telephone recordings, etc.) into text, and the document extraction plug-in is used to automatically extract and analyze legal documents to help criminal investigators understand the legal background of related cases, case references, etc. The data visualization plug-in is used to graphically and visually display various information in the case (such as suspect information, case progress, crime trends, etc.).

[0152] It should be understood that after completing the workflow design, human-computer interaction interface design and preset auxiliary plug-in development, the target criminal investigation workflow, preset human-computer interaction interface and preset auxiliary plug-in can be integrated into the intelligent agent using the intelligent agent construction tool to obtain a criminal investigation intelligent agent.

[0153] This embodiment provides a method for constructing an intelligent criminal investigation platform, creating a criminal investigation workflow, the criminal investigation workflow includes a pattern recognition workflow, a legal consultation workflow, and a report generation workflow; based on a retrieval enhancement strategy, the criminal investigation big model and the criminal investigation knowledge vector database are referenced in the criminal investigation workflow to obtain a target criminal investigation workflow; a criminal investigation intelligent body is constructed based on the target criminal investigation workflow, a preset human-computer interaction interface, and a preset auxiliary plug-in. By combining the pattern recognition workflow, the legal consultation workflow, and the report generation workflow to define the operation logic of the criminal investigation intelligent body, the process of criminal investigation work is optimized, and based on the retrieval enhancement strategy, the criminal investigation big model and the criminal investigation knowledge vector database are referenced, which effectively improves the efficiency and accuracy of criminal investigation work and enhances the real-time and precision of decision support.

[0154] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the method for constructing the intelligent criminal investigation platform of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0155] This application also provides a construction device for an intelligent criminal investigation platform, please refer to Figure 5 , the construction device of the intelligent criminal investigation platform includes:

[0156] A model fine-tuning module 10 is used to obtain a multimodal criminal investigation dataset, and fine-tune the generative large language model based on the multimodal criminal investigation dataset to obtain a criminal investigation large model;

[0157] The knowledge construction module 20 is used to construct a criminal investigation knowledge base based on criminal investigation knowledge data through a text analysis strategy, and store the criminal investigation knowledge base in a vector database to obtain a criminal investigation knowledge vector database;

[0158] An agent building module 30, for building a criminal investigation agent based on the criminal investigation big model and the criminal investigation knowledge vector database in combination with a search enhancement strategy, wherein the criminal investigation agent is used to respond to a criminal investigation question and answer request from a user;

[0159] The platform construction module 40 is used to deploy the criminal investigation agent to the business application layer to complete the construction of the intelligent criminal investigation platform.

[0160] In one embodiment, the knowledge construction module 20 is further used to preprocess the criminal investigation knowledge data through a text parsing strategy to obtain a preprocessed text; perform entity recognition on the preprocessed text based on criminal investigation entity category information to obtain a criminal investigation word vector sequence; perform relationship extraction on the criminal investigation word vector sequence to obtain a criminal investigation relationship sequence; construct a criminal investigation knowledge base with the criminal investigation word vector sequence and the criminal investigation relationship sequence; convert the criminal investigation knowledge base into an embedded vector and store it in a vector database to obtain a criminal investigation knowledge vector database.

[0161] In one embodiment, the knowledge construction module 20 is also used to obtain a position index sequence and a dimension index sequence of the criminal investigation word vector sequence; based on the position index sequence and the dimension index sequence, the position information is integrated into the criminal investigation word vector sequence to obtain a first word vector sequence; based on the first word vector sequence, criminal investigation word pair features are calculated to obtain a first criminal investigation word pair feature matrix; based on the criminal investigation entity category information, an entity category information matrix is ​​generated; the entity category information matrix is ​​concatenated with the first criminal investigation word pair feature matrix to obtain a second criminal investigation word pair feature matrix; the horizontal axis attention and vertical axis attention of the second criminal investigation word pair feature matrix are calculated to obtain an axial attention matrix; the axial attention matrix and the second criminal investigation word pair feature matrix are residually connected to obtain a criminal investigation relationship sequence.

[0162] In one embodiment, the model fine-tuning module 10 is also used to load a pre-trained generative large language model, and input the multimodal criminal investigation dataset into the generative large language model, the generative large language model includes a multimodal encoder; the data of the multimodal criminal investigation dataset is encoded by the multimodal encoder to obtain a criminal investigation image feature vector, a criminal investigation text feature vector and a criminal investigation label vector; the cosine similarity is calculated according to the criminal investigation image feature vector, the criminal investigation text feature vector and the criminal investigation label vector; the multimodal encoder is optimized according to the cosine similarity to minimize the similarity loss between the criminal investigation image feature vector and the criminal investigation text feature vector, and obtain a criminal investigation large model.

[0163] In one embodiment, the model fine-tuning module 10 is further used to assign cross-modal attention weights to the criminal investigation image feature vector and the criminal investigation text feature vector based on the criminal investigation label vector; weight the criminal investigation image feature vector and the criminal investigation text feature vector based on the cross-modal attention weights to obtain a common feature vector; calculate the inner product of a preset hyperplane and the common feature vector to obtain a binary hash code sequence; determine a cross-modal hash code based on the hash value sequence; and calculate the cosine similarity of the cross-modal hash code.

[0164] In one embodiment, the intelligent agent construction module 30 is also used to create a criminal investigation workflow, which includes a pattern recognition workflow, a legal consultation workflow, and a report generation workflow; based on a retrieval enhancement strategy, the criminal investigation big model and the criminal investigation knowledge vector database are referenced in the criminal investigation workflow to obtain a target criminal investigation workflow; and a criminal investigation intelligent agent is constructed based on the target criminal investigation workflow, a preset human-computer interaction interface, and a preset auxiliary plug-in.

[0165] In one embodiment, the model fine-tuning module 10 is also used to collect criminal investigation text data and criminal investigation image data; perform data cleaning on the criminal investigation text data and the criminal investigation image data to obtain pre-processed criminal investigation data; and perform data labeling on the pre-processed criminal investigation data to obtain a multimodal criminal investigation data set.

[0166] The construction device of the intelligent criminal investigation platform provided by the present application adopts the construction method of the intelligent criminal investigation platform in the above-mentioned embodiment, which can solve the technical problem of how to construct an intelligent criminal investigation platform to optimize the criminal investigation process and improve the efficiency of criminal investigation work. Compared with the prior art, the beneficial effects of the construction device of the intelligent criminal investigation platform provided by the present application are the same as the beneficial effects of the construction method of the intelligent criminal investigation platform provided by the above-mentioned embodiment, and the other technical features in the construction device of the intelligent criminal investigation platform are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.

[0167] The present application provides a device for constructing an intelligent criminal investigation platform, and the device for constructing the intelligent criminal investigation platform includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for constructing the intelligent criminal investigation platform in the above-mentioned embodiment one.

[0168] Reference below Figure 6 , which shows a schematic diagram of the structure of the construction equipment of the intelligent criminal investigation platform suitable for implementing the embodiment of the present application. The construction equipment of the intelligent criminal investigation platform in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The construction device of the intelligent criminal investigation platform shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0169] like Figure 6As shown, the construction device of the intelligent criminal investigation platform may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 to the random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the construction device of the intelligent criminal investigation platform are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the construction device of the intelligent criminal investigation platform to communicate wirelessly or wired with other devices to exchange data. Although the construction device of the intelligent criminal investigation platform with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0170] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0171] The construction device of the intelligent criminal investigation platform provided by the present application adopts the construction method of the intelligent criminal investigation platform in the above embodiment, which can solve the technical problem of how to construct an intelligent criminal investigation platform to optimize the criminal investigation process and improve the efficiency of criminal investigation work. Compared with the prior art, the beneficial effects of the construction device of the intelligent criminal investigation platform provided by the present application are the same as the beneficial effects of the construction method of the intelligent criminal investigation platform provided by the above embodiment, and the other technical features in the construction device of the intelligent criminal investigation platform are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0172] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0173] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0174] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the method for constructing the intelligent criminal investigation platform in the above-mentioned embodiment.

[0175] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0176] The above-mentioned computer-readable storage medium may be included in the construction device of the intelligent criminal investigation platform; or it may exist independently without being assembled into the construction device of the intelligent criminal investigation platform.

[0177] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the construction device of the intelligent criminal investigation platform, the construction device of the intelligent criminal investigation platform: obtains a multimodal criminal investigation data set, and fine-tunes the generative large language model based on the multimodal criminal investigation data set to obtain a criminal investigation large model; constructs a criminal investigation knowledge base based on criminal investigation knowledge data through a text parsing strategy, and stores the criminal investigation knowledge base in a vector database to obtain a criminal investigation knowledge vector database; constructs a criminal investigation intelligent body based on the criminal investigation large model and the criminal investigation knowledge vector database in combination with a retrieval enhancement strategy, and the criminal investigation intelligent body is used to respond to the user's criminal investigation question and answer request; and deploys the criminal investigation intelligent body to the business application layer to complete the construction of the intelligent criminal investigation platform.

[0178] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0179] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0180] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0181] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the method for constructing the above-mentioned intelligent criminal investigation platform, and can solve the technical problem of how to construct an intelligent criminal investigation platform to optimize the criminal investigation process and improve the efficiency of criminal investigation work. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the method for constructing the intelligent criminal investigation platform provided in the above-mentioned embodiment, and will not be repeated here.

[0182] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the method for constructing the intelligent criminal investigation platform as described above.

[0183] The computer program product provided by this application can solve the technical problem of how to build an intelligent criminal investigation platform to optimize the criminal investigation process and improve the efficiency of criminal investigation work. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the method for building an intelligent criminal investigation platform provided by the above embodiment, and will not be repeated here.

[0184] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for constructing an intelligent criminal investigation platform, characterized in that: The method for constructing the intelligent criminal investigation platform includes: Acquire a multimodal criminal investigation dataset, and fine-tune and train a generative large language model based on the multimodal criminal investigation dataset to obtain a criminal investigation large model; Building a criminal investigation knowledge base based on criminal investigation knowledge data through a text parsing strategy, and storing the criminal investigation knowledge base in a vector database to obtain a criminal investigation knowledge vector database; The steps of constructing a criminal investigation knowledge base based on criminal investigation knowledge data through a text parsing strategy, and storing the criminal investigation knowledge base in a vector database to obtain a criminal investigation knowledge vector database include: preprocessing the criminal investigation knowledge data through a text parsing strategy to obtain a preprocessed text; performing entity recognition on the preprocessed text based on criminal investigation entity category information to obtain a criminal investigation word vector sequence; performing relationship extraction on the criminal investigation word vector sequence to obtain a criminal investigation relationship sequence; constructing a criminal investigation knowledge base with the criminal investigation word vector sequence and the criminal investigation relationship sequence; converting the criminal investigation knowledge base into an embedded vector and storing it in a vector database to obtain a criminal investigation knowledge vector database; The step of performing relationship extraction on the criminal investigation word vector sequence to obtain a criminal investigation relationship sequence includes: obtaining a position index sequence and a dimension index sequence of the criminal investigation word vector sequence; integrating position information into the criminal investigation word vector sequence based on the position index sequence and the dimension index sequence to obtain a first word vector sequence; calculating criminal investigation word pair features based on the first word vector sequence to obtain a first criminal investigation word pair feature matrix; generating an entity category information matrix based on the criminal investigation entity category information; concatenating the entity category information matrix with the first criminal investigation word pair feature matrix to obtain a second criminal investigation word pair feature matrix; calculating the horizontal axis attention and the vertical axis attention of the second criminal investigation word pair feature matrix to obtain an axial attention matrix; performing residual connection on the axial attention matrix and the second criminal investigation word pair feature matrix to obtain a criminal investigation relationship sequence; The formula for calculating the features of criminal investigation word pairs is as follows: in, and are the i-th criminal investigation word vector and the j-th criminal investigation word vector in the first word vector sequence, respectively. W is a learnable weight matrix. is a bias term of dimension d×d, where d is the dimension of the criminal investigation word vector. is the activation function, For criminal investigation word vector and The criminal investigation word pair features obtained by calculation; The step of integrating the position information into the criminal investigation word vector sequence based on the position index sequence and the dimension index sequence to obtain a first word vector sequence includes: calculating the position embedding vector of the criminal investigation word vector at each position and each dimension in the criminal investigation word vector sequence based on the position index sequence and the dimension index sequence; obtaining a position vector matrix according to the position embedding vector; splicing the criminal investigation word vector sequence and the position vector matrix to obtain a first word vector sequence incorporating the position information; Based on the criminal investigation big model and the criminal investigation knowledge vector database, a criminal investigation intelligent agent is constructed in combination with a search enhancement strategy, and the criminal investigation intelligent agent is used to respond to the criminal investigation question and answer request of the user; The criminal investigation agent is deployed to the business application layer to complete the construction of the intelligent criminal investigation platform.

2. The method according to claim 1, characterized in that The step of fine-tuning and training the generative large language model based on the multimodal criminal investigation dataset to obtain the criminal investigation large model includes: Loading a pre-trained generative large language model, and inputting the multimodal criminal investigation dataset into the generative large language model, wherein the generative large language model includes a multimodal encoder; Encoding the data of the multimodal criminal investigation dataset by the multimodal encoder to obtain a criminal investigation image feature vector, a criminal investigation text feature vector and a criminal investigation label vector; Calculating cosine similarity based on the criminal investigation image feature vector, the criminal investigation text feature vector, and the criminal investigation label vector; The multimodal encoder is optimized according to the cosine similarity to minimize the similarity loss between the criminal investigation image feature vector and the criminal investigation text feature vector, thereby obtaining a criminal investigation large model.

3. The method according to claim 2, characterized in that The step of calculating the cosine similarity based on the criminal investigation image feature vector, the criminal investigation text feature vector and the criminal investigation label vector comprises: assigning cross-modal attention weights to the criminal investigation image feature vector and the criminal investigation text feature vector based on the criminal investigation label vector; weighting the criminal investigation image feature vector and the criminal investigation text feature vector based on the cross-modal attention weight to obtain a common feature vector; Calculating the inner product of a preset hyperplane and the common feature vector to obtain a binary hash code sequence; Determining a cross-modal hash code based on the binary hash code sequence; The cosine similarity of the cross-modal hash codes is calculated.

4. The method according to claim 1, characterized in that The step of constructing a criminal investigation intelligent agent based on the criminal investigation big model and the criminal investigation knowledge vector database in combination with a retrieval enhancement strategy includes: Creating a criminal investigation workflow, wherein the criminal investigation workflow includes a pattern recognition workflow, a legal consultation workflow, and a report generation workflow; Based on the retrieval enhancement strategy, the criminal investigation big model and the criminal investigation knowledge vector database are referenced in the criminal investigation workflow to obtain a target criminal investigation workflow; A criminal investigation intelligent agent is constructed based on the target criminal investigation workflow, the preset human-computer interaction interface and the preset auxiliary plug-in.

5. The method according to any one of claims 1 to 4, characterized in that: The step of obtaining a multimodal criminal investigation dataset includes: Collect criminal investigation text data and criminal investigation image data; Performing data cleaning on the criminal investigation text data and the criminal investigation image data to obtain pre-processed criminal investigation data; The pre-processed criminal investigation data is annotated to obtain a multimodal criminal investigation data set.

6. A device for constructing an intelligent criminal investigation platform, characterized in that: The device comprises: A model fine-tuning module, used to obtain a multimodal criminal investigation dataset, and fine-tune the generative large language model based on the multimodal criminal investigation dataset to obtain a criminal investigation large model; A knowledge construction module is used to construct a criminal investigation knowledge base based on criminal investigation knowledge data through a text analysis strategy, and store the criminal investigation knowledge base in a vector database to obtain a criminal investigation knowledge vector database; The knowledge construction module is further used to pre-process the criminal investigation knowledge data through a text parsing strategy to obtain a pre-processed text; perform entity recognition on the pre-processed text based on criminal investigation entity category information to obtain a criminal investigation word vector sequence; perform relationship extraction on the criminal investigation word vector sequence to obtain a criminal investigation relationship sequence; construct a criminal investigation knowledge base with the criminal investigation word vector sequence and the criminal investigation relationship sequence; convert the criminal investigation knowledge base into an embedded vector and store it in a vector database to obtain a criminal investigation knowledge vector database; The knowledge construction module is also used to obtain the position index sequence and dimension index sequence of the criminal investigation word vector sequence; based on the position index sequence and the dimension index sequence, the position information is integrated into the criminal investigation word vector sequence to obtain a first word vector sequence; based on the first word vector sequence, criminal investigation word pair features are calculated to obtain a first criminal investigation word pair feature matrix; based on the criminal investigation entity category information, an entity category information matrix is ​​generated; the entity category information matrix is ​​concatenated with the first criminal investigation word pair feature matrix to obtain a second criminal investigation word pair feature matrix; the horizontal axis attention and the vertical axis attention of the second criminal investigation word pair feature matrix are calculated to obtain an axial attention matrix; the axial attention matrix and the second criminal investigation word pair feature matrix are residually connected to obtain a criminal investigation relationship sequence; The formula for calculating the features of criminal investigation word pairs is as follows: in, and are the i-th criminal investigation word vector and the j-th criminal investigation word vector in the first word vector sequence, respectively. W is a learnable weight matrix. is a bias term of dimension d×d, where d is the dimension of the criminal investigation word vector. is the activation function, For criminal investigation word vector and The criminal investigation word pair features obtained by calculation; The knowledge construction module is further used to calculate the position embedding vector of the criminal investigation word vector at each position and each dimension in the criminal investigation word vector sequence based on the position index sequence and the dimension index sequence; obtain a position vector matrix according to the position embedding vector; and concatenate the criminal investigation word vector sequence and the position vector matrix to obtain a first word vector sequence incorporating position information; An intelligent agent building module, used to build a criminal investigation intelligent agent based on the criminal investigation big model and the criminal investigation knowledge vector database in combination with a retrieval enhancement strategy, wherein the criminal investigation intelligent agent is used to respond to a criminal investigation question and answer request from a user; The platform construction module is used to deploy the criminal investigation agent to the business application layer to complete the construction of the intelligent criminal investigation platform.

7. A device for constructing an intelligent criminal investigation platform, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the method for constructing an intelligent criminal investigation platform as described in any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the method for constructing an intelligent criminal investigation platform as described in any one of claims 1 to 5 are implemented.

Images