Data writing method and device, storage medium and electronic device

Abstract

Embodiments of the present application provide a data writing method and device, a storage medium and an electronic device. The method comprises: obtaining an output result output by a target model after analyzing a target data set received by the target model, the target data set being data received in a current predetermined time period and to be written into a target hard disk, the output result including a current writing stage for indicating that data is currently being written into the target hard disk; determining a target writing mode of the target data set based on the current writing stage; and writing the target data set into the target hard disk according to the target writing mode. Through the present application, the problem of excessive storage pressure of the storage medium in the related art, which leads to low data writing efficiency, is solved, and the effect of improving the data writing efficiency and the data storage reliability is achieved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of communication, and in particular, to a data writing method and device, a storage medium and an electronic device. BACKGROUND

[0002] With the rapid development of science and technology and the improvement of people's living standards, electronic products are everywhere in daily life, so people's experience requirements for electronic products are also getting higher and higher, and the demand for electronic products that can store data faster and more conveniently is also increasing.

[0003] In related technologies, data storage is usually performed by using the high read-write performance of solid state disk flash memory, but because the solid state disk has the characteristic of being unable to be rewritten, the effective data included in the data stored in the solid state disk needs to be moved and erased before the next write operation can be performed. In this case, subsequent write operations will reduce the read-write performance of the solid state disk, thereby affecting the data writing efficiency.

[0004] At present, there is no effective solution to the problem of excessive storage pressure of storage media in related technologies, which leads to low data writing efficiency. SUMMARY

[0005] Embodiments of the present application provide a data writing method and device, a storage medium and an electronic device to at least solve the problem of excessive storage pressure of storage media in related technologies, which leads to low data writing efficiency.

[0006] According to an embodiment of the present application, a data writing method is provided, comprising: obtaining an output result output by a target model after analyzing a target data set received, wherein the target data set is data received in a current predetermined time period to be written into a target hard disk, and the output result includes a current writing stage for indicating that data is currently being written into the target hard disk; determining a target writing mode of the target data set based on the current writing stage; and writing the target data set into the target hard disk according to the target writing mode.

[0007] In an example embodiment, the target model is configured to analyze the target dataset and output the output result by: determining the current write stage included in the output result as a first stage, if it is determined that the data amount of the target dataset is less than or equal to a first data amount threshold; determining the current write stage included in the output result as one of a second stage and a fourth stage, if it is determined that the data amount of the target dataset is greater than the first data amount threshold and less than or equal to a second data amount threshold; and determining the current write stage included in the output result as a third stage, if it is determined that the data amount of the target dataset is greater than the second data amount threshold.

[0008] In an example embodiment, determining the target write mode of the target dataset based on the current write stage comprises: determining the target write mode as a multi-layer storage unit direct write mode, if the current write stage is the first stage, the second stage or the fourth stage; and determining the target write mode as a single-layer storage unit write mode, if the current write stage is the third stage.

[0009] In an example embodiment, the target model is further configured to: compare the data amount of the target dataset with a data amount of a first dataset, if it is determined that the data amount of the target dataset is greater than the first data amount threshold and less than or equal to the second data amount threshold; determine the current write stage as the second stage, if it is determined that the data amount of the target dataset is greater than the data amount of the first dataset; and determine the current write stage as the fourth stage, if it is determined that the data amount of the target dataset is less than the data amount of the first dataset; wherein the first dataset is a dataset received by the target model before receiving the target dataset.

[0010] In an example embodiment, if the current write stage included in the output result is the second stage, the output result further includes a predicted data amount of data to be written into the target hard disk in the third stage, and the method further comprises: determining a first number of available blocks needed by the target hard disk to store the predicted data amount in the third stage based on the predicted data amount, wherein the available blocks are first storage blocks included in the target hard disk and not occupied by other data; determining a second number of the available blocks included in the target hard disk before entering the third stage; and determining target available blocks needed by the target hard disk to store the predicted data amount in the third stage based on the first number and the second number.

[0011] In an example embodiment, determining the target available blocks needed by the target hard disk to store the predicted data amount in the third stage based on the first quantity and the second quantity comprises: comparing the first quantity with the second quantity; in a case where it is determined that the first quantity is greater than the second quantity, recycling second storage blocks included in the target hard disk and occupied by other data according to a target recycling manner, and determining the target available blocks as the second quantity of the available blocks and the recycled second storage blocks; in a case where it is determined that the first quantity is less than or equal to the second quantity, determining the first quantity of the available blocks from the second quantity of the available blocks, and determining the target available blocks as the first quantity of the available blocks.

[0012] In an example embodiment, in a case where it is determined that the first quantity is greater than the second quantity, recycling second storage blocks included in the target hard disk and occupied by other data according to a target recycling manner comprises at least one of the following manners: manner one: recycling the second storage blocks included in the target hard disk and occupied by the other data whose valid data proportion is less than a predetermined threshold; manner two: determining a quantity N1 of first candidate storage blocks to be screened according to the following formula:

[0013] N1>g*a1+ε

[0014] wherein g is a first predetermined parameter, ε is a second predetermined parameter, a1 is the predicted data amount divided by a capacity of a single storage block included in the target hard disk, and a1 is the predicted data amount divided by a capacity of a single storage block included in the target hard disk; determining N first candidate storage blocks from the target hard disk according to a timestamp indicating data storage, and recycling the second storage blocks from the N first candidate storage blocks.

[0015] In an example embodiment, recycling the second storage blocks from the N candidate storage blocks comprises: sorting the N candidate storage blocks in a sequence from low to high according to a read frequency of the N candidate storage blocks; and recycling the second storage blocks from the N candidate storage blocks according to the sorting sequence.

[0016] In an example embodiment, in a case where a third quantity of the second storage blocks recycled according to the manner one is less than a first difference, wherein the first difference is a difference between the first quantity and the second quantity, the method further comprises: determining a quantity N2 of second candidate storage blocks to be screened according to the following formula:

[0017] N2>g*a2+ε

[0018] The a2 is a difference between the first difference and the third quantity; and N2 second candidate storage blocks are determined from the target hard disk according to a time stamp used for indicating data storage, and a2 second storage blocks are recovered from the N2 second candidate storage blocks.

[0019] In one example embodiment, before the output result output by the target model after analyzing the received target data set is acquired, the method further comprises: acquiring the target data set obtained by the target monitoring device photographing the target object in the target area within the current predetermined time period; and inputting the target data set into the target model to instruct the target model to analyze the target data set.

[0020] According to another embodiment of the present application, a data writing device is provided, comprising: a first acquisition module configured to acquire an output result output by a target model after analyzing a target data set received by the target model, wherein the target data set is data received in a current predetermined time period and to be written into a target hard disk, and the output result includes a current writing stage used for indicating that data is currently being written into the target hard disk; a first determination module configured to determine a target writing mode of the target data set based on the current writing stage; and a writing module configured to write the target data set into the target hard disk according to the target writing mode.

[0021] According to still another embodiment of the present application, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0022] According to still another embodiment of the present application, an electronic device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to execute the steps in any of the above method embodiments.

[0023] By the present application, the target model is used to analyze the target data set received in a predetermined time period, and the output result of the current writing stage for indicating the current writing data in the target hard disk is obtained, and then the target writing mode of the target data set is determined based on the current writing stage, and the target data set is written into the target hard disk according to the target writing mode. By using the method of the present application, the target model is used to analyze the target data set received in a predetermined time period to determine the current writing stage of the current writing data in the target hard disk, and then the target writing mode of the target data set is dynamically adjusted based on the current writing stage, so as to improve the storage performance of the storage medium by dynamically adjusting the target writing mode of the target data set, and solve the problem of low data writing efficiency caused by excessive storage pressure of the storage medium in the related art, and achieve the effect of improving the data writing efficiency and the data storage reliability. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is a hardware structure block diagram of a mobile terminal of a data writing method according to an embodiment of the present application;

[0025] Figure 2 is a flowchart of a data writing method according to an embodiment of the present application;

[0026] Figure 3 is a curve diagram of a data writing stage according to a specific embodiment of the present application;

[0027] Figure 4 is a flowchart of available block recycling according to a specific embodiment of the present application;

[0028] Figure 5 is a structure block diagram of a data writing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] Hereinafter, the embodiments of the present application will be described in detail with reference to the accompanying drawings and in conjunction with the embodiments.

[0030] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0031] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking the case of running on a mobile terminal, Figure 1 is a hardware structure block diagram of a mobile terminal of a data writing method according to an embodiment of the present application. As shown in Figure 1 , the mobile terminal can include one or more Figure 1The mobile terminal shown in FIG. 1 includes only one processor 102 (the processor 102 can include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), and a memory 104 for storing data, wherein the mobile terminal can further include a transmission device 106 for communication function and an input / output device 108. Those skilled in the art can understand that, Figure 1 The structure shown is only schematic, and does not limit the structure of the mobile terminal. For example, the mobile terminal can include more or less components than those shown, or have a different configuration of components than those shown. Figure 1 The mobile terminal shown in FIG. 1 includes only one processor 102 (the processor 102 can include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), and a memory 104 for storing data, wherein the mobile terminal can further include a transmission device 106 for communication function and an input / output device 108. Those skilled in the art can understand that, Figure 1 The mobile terminal shown in FIG. 1 includes only one processor 102 (the processor 102 can include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), and a memory 104 for storing data, wherein the mobile terminal can further include a transmission device 106 for communication function and an input / output device 108. Those skilled in the art can understand that,

[0032] The memory 104 can be used to store computer programs, such as software programs of application software and modules, such as a computer program corresponding to the data writing method in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, that is, implements the above-mentioned method. The memory 104 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, a flash memory, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely arranged with respect to the processor 102, and the remote memory can be connected to the mobile terminal through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0033] The transmission device 106 is used to receive or send data via a network. Specific examples of the network can include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC for short), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, RF for short) module, which is used to communicate with the Internet in a wireless manner.

[0034] In the present embodiment, a data writing method is provided, Figure 2 The flowchart of the data writing method according to the embodiments of the present application is shown in FIG. 2, which includes the following steps: Figure 2

[0035] S202, obtaining an output result output by a target model after analyzing a target data set, wherein the target data set is data received in a current predetermined time period and to be written into a target hard disk, and the output result includes a current writing stage for indicating that data is currently being written into the target hard disk; ​

[0036] S204, determining a target writing mode of the target data set based on the current writing stage;

[0037] S206, writing the target data set into the target hard disk according to the target writing mode.

[0038] Wherein, the controller or the control system, or the device with control and storage capabilities, such as solid state disk integrated with control and storage capabilities, or control device connected with solid state disk, or processor or controller arranged in the device, or other processing devices or processing units with similar processing capabilities, etc., can perform the above operations.

[0039] In the above embodiments, there can be multiple target models, which can be used to analyze the received target data set at the same time, and then multiple output results output by the multiple target models after analyzing the same target data set are obtained, and the multiple output results are further compared to determine whether the analysis results of the multiple target models are accurate. For example, when there are three target models (target model 1, target model 2, and target model 3) that analyze the received target data set at the same time, and three output results (output result 1, output result 2, and output result 3) are output, the output result 1 output by the target model 1 is consistent with the output result 3 output by the target model 3, but inconsistent with the output result 2 output by the target model 2, which indicates that the target model 2 may have problems such as inaccurate data processing parameters or unreasonable algorithm design, and the target model 2 needs to be further detected. At this time, the output result 1 output by the target model 1 or the output result 3 output by the target model 3 can be taken as the correct analysis result, or the output result 1 output by the target model 1 and the output result 3 output by the target model 3 can be averaged to take the average result as the correct analysis result. In addition, the priority of the target model can be used to determine the analysis result. For example, when there are three target models (target model 1, target model 2, and target model 3) that analyze the received target data set at the same time, and three output results (output result 1, output result 2, and output result 3) are output, the target model priorities are target model 1> target model 2> target model 3, wherein the output result 1 output by the target model 1 is inconsistent with the output result 2 output by the target model 2 and the output result 3 output by the target model 3, which indicates that the target model 2 and the target model 3 may have problems such as inaccurate data processing parameters or unreasonable algorithm design, and the target model 2 and the target model 3 need to be further detected. At this time, the output result 1 output by the target model 1 can be taken as the correct analysis result, and so on. It should be noted that the above examples of target models and output results are only exemplary embodiments, and the target models and output results are not limited to the above examples.

[0040] In the above embodiment, the predetermined time period can be set in advance, which can be set to 5 seconds, 15 seconds, 30 seconds, 1 minute, etc. For example, when the predetermined time period is 15 seconds, the target data set is the data received in the current 15 seconds and to be written into the target hard disk, and then the output result output by the target model after analyzing the target data set received in the current 15 seconds and to be written into the target hard disk can be obtained. It should be noted that the above example of the predetermined time period is only an example, and the example of the predetermined time period is not limited to the above example.

[0041] In the above embodiment, the output result output by the target model after analyzing the target data set received in the current predetermined time period and to be written into the target hard disk, which is used to indicate the current writing stage of writing data in the target hard disk, is obtained, and then the target writing mode of the target data set is determined based on the current writing stage, and then the target data set is written into the target hard disk according to the target writing mode. By using the method, the target model is used to analyze the target data set received in the current predetermined time period and to be written into the target hard disk to determine the current writing stage of writing data in the target hard disk, and then the target writing mode of the target data set is dynamically adjusted based on the current writing stage, so that the storage performance of the storage medium is improved by dynamically adjusting the target writing mode of the target data set, the problem of low data writing efficiency caused by excessive storage pressure of the storage medium in the related art is solved, and the effects of improving the data writing efficiency and the data storage reliability are achieved.

[0042] In an example embodiment, the target model is used to analyze the target data set and output the output result by: determining the current writing stage included in the output result as a first stage in a case that a data amount of the target data set is determined to be less than or equal to a first data amount threshold; determining the current writing stage included in the output result as one of a second stage and a fourth stage in a case that the data amount of the target data set is determined to be greater than the first data amount threshold and less than or equal to a second data amount threshold; and determining the current writing stage included in the output result as a third stage in a case that the data amount of the target data set is determined to be greater than the second data amount threshold. In this embodiment, the first data amount threshold and the second data amount threshold can be pre-configured, the first data amount threshold can be 50T, 100T, 150T, etc., and the second data amount threshold can be 200T, 250T, 300T, etc. For example, when the first data amount threshold is 100T and the second data amount threshold is 200T, the current writing stage included in the output result is determined to be the first stage in a case that the data amount of the target data set is determined to be less than or equal to 100T, the current writing stage included in the output result is determined to be one of the second stage and the fourth stage in a case that the data amount of the target data set is determined to be greater than 100T and less than or equal to 200T, and the current writing stage included in the output result is determined to be the third stage in a case that the data amount of the target data set is determined to be greater than 200T. It should be noted that the above example of the first data amount threshold and the second data amount threshold is only an example embodiment, and the first data amount threshold and the second data amount threshold are not limited to the above example. In addition, it should be noted that the stages of the data writing process are generally divided into four stages, but in actual applications, more stages (e.g., five stages, seven stages, etc.) or fewer stages (e.g., three stages) can be divided according to actual needs, and each stage can be pre-configured with a most suitable data writing mode, and different stages can correspond to specific writing data amount thresholds. The specific data writing process is similar to the above and will not be described here.

[0043] In one example embodiment, determining the target writing mode of the target data set based on the current writing stage includes: in the case that the current writing stage is the first stage, the second stage or the fourth stage, determining the target writing mode as the multi-layer storage unit direct writing mode; in the case that the current writing stage is the third stage, determining the target writing mode as the single-layer storage unit writing mode. In this embodiment, the current writing stage is determined according to the size of the data amount of the target data set, and then the target data set is written into the target hard disk by using the multi-layer storage unit direct writing mode or the single-layer storage unit writing mode according to the current writing stage. In addition, in one optional embodiment, there can be a stage that uses both target writing modes at the same time. For example, when the data amount of the target data set is 400T, part of the data (for example, 200T) can be written into the target hard disk by using the multi-layer storage unit direct writing mode, and the remaining data (for example, the remaining 200T) can be written into the target hard disk by using the single-layer storage unit writing mode, and so on. In addition, the multi-layer storage unit direct writing mode includes but is not limited to the TLC (Trinary-Level Cell) direct writing mode, the MLC (Multi-Level Cell) direct writing mode, and the like. It should be noted that the above example of the target writing mode is only one example embodiment, and the target writing mode is not limited to the above examples.

[0044] In the above embodiment, the single-layer storage unit writing mode is to write the target data set into the single-layer storage unit block with faster speed for storage, thereby improving the writing efficiency of the data, and in the case that the storage pressure of the target hard disk is small, the target data set stored in the single-layer storage unit block is GC (Garbage Collection) to the multi-layer storage unit block, and the multi-layer storage unit direct writing mode is to directly write the target data set into the multi-layer storage unit block without GC processing of the target data set stored in the multi-layer storage unit block.

[0045] In an example embodiment, the target model is further configured to compare the data amount of the target data set with a data amount of a first data set in a case where it is determined that the data amount of the target data set is greater than a first data amount threshold and less than or equal to a second data amount threshold; determine that the current write stage is the second stage in a case where it is determined that the data amount of the target data set is greater than the data amount of the first data set; and determine that the current write stage is the fourth stage in a case where it is determined that the data amount of the target data set is less than the data amount of the first data set. In this embodiment, when the data amount of the target data set is greater than the data amount of the first data set, it indicates that the data amount of the data continuously received for writing into the target hard disk is developing from less to more, i.e., the current write stage is the second stage. When the data amount of the target data set is less than the data amount of the first data set, it indicates that the data amount of the data continuously received for writing into the target hard disk is developing from more to less, i.e., the current write stage is the fourth stage.

[0046] In an example embodiment, in a case where the current write stage included in the output result is the second stage, the output result further includes a predicted data amount of the data to be written into the target hard disk in the third stage. The method further includes determining a first number of available blocks needed by the target hard disk to store the predicted data amount in the third stage based on the predicted data amount, wherein the available blocks are first storage blocks included in the target hard disk and not occupied by other data; determining a second number of the available blocks included in the target hard disk before entering the third stage; and determining target available blocks needed by the target hard disk to store the predicted data amount in the third stage based on the first number and the second number. In this embodiment, the target model can be used to predict the predicted data amount of the data to be written into the target hard disk in the third stage in addition to determining the current write stage based on the data amount of the target data set. By predicting the data amount of the data to be written into the target hard disk in the third stage (e.g., a large data amount stage) in advance, the storage resources in the target hard disk can be reasonably arranged and scheduled in a case where the storage resources in the target hard disk are sufficient, and the corresponding storage resources can be prepared in advance through certain strategies in a case where the storage resources in the target hard disk are insufficient, thereby achieving the purpose of improving the data write efficiency in the third stage.

[0047] In one example embodiment, determining the target available blocks needed by the target hard disk to store the predicted data amount in the third stage based on the first quantity and the second quantity comprises: comparing the first quantity with the second quantity; in a case where it is determined that the first quantity is greater than the second quantity, recovering the second storage blocks included in the target hard disk and occupied by the other data according to a target recovery mode, and determining the target available blocks as the second quantity of the available blocks and the recovered second storage blocks; in a case where it is determined that the first quantity is less than or equal to the second quantity, determining the first quantity of the available blocks from the second quantity of the available blocks, and determining the target available blocks as the determined first quantity of the available blocks. In this embodiment, in a case where the first quantity is greater than the second quantity and the difference between the first quantity and the second quantity exceeds a first predetermined quantity, the second storage blocks included in the target hard disk and occupied by the other data can be recovered according to the target recovery mode; in a case where the first quantity is greater than the second quantity and the difference between the first quantity and the second quantity does not exceed the first predetermined quantity, the target available blocks are determined as the second quantity of the available blocks and the first predetermined quantity of the available blocks, wherein the first predetermined quantity of the available blocks are the spare storage blocks reserved in the target hard disk. For example, when the target hard disk has a total of 6 available blocks, 2 available blocks can be reserved in advance as spare available blocks; when the target data is written into the target hard disk and occupies 4 available blocks, and there is still some data not written into the target hard disk, the data can be stored in the spare available blocks, so as to ensure the data storage efficiency in a short time and provide some buffer time for the subsequent recovery process.

[0048] In one example embodiment, in a case where it is determined that the first quantity is greater than the second quantity, recovering the second storage blocks included in the target hard disk and occupied by the other data according to the target recovery mode comprises at least one of the following modes: mode one: recovering the second storage blocks in which the proportion of valid data in the other data included in the target hard disk is less than a predetermined threshold; mode two: determining the number N1 of the first candidate storage blocks to be screened according to the following formula:

[0049] N1>g*a1+ε

[0050] Wherein, the g is a first predetermined parameter, the ε is a second predetermined parameter, the a1 is the first quantity, the a1 is the predicted data quantity divided by the capacity of a single storage block included in the target hard disk; N pieces of the first candidate storage blocks are determined from the target hard disk according to the time stamp for indicating data storage, and the second storage block is recovered from the N pieces of the first candidate storage blocks. In the embodiment, the predetermined threshold can be a predetermined setting, which can be set to 0.1, 0.2, 0.3, etc. For example, when the predetermined threshold is 0.2, the second storage block with an effective data proportion less than 0.2 in the other data included in the target hard disk is recovered. It should be noted that the above example of the predetermined threshold is only an example, and the predetermined threshold is not limited to the above example.

[0051] In the above embodiment, the first predetermined parameter and the second predetermined parameter can be pre-set. The first predetermined parameter can be set to 1.4, 1.5, 1.6, etc., and the second predetermined parameter can be set to 5, 6, 7, etc. For example, when the first predetermined parameter is 1.5, the second predetermined parameter is 5, and the predicted data quantity is 200T, the number of the first candidate storage blocks N1 determined by the formula N1>g*a1+ε is greater than 305. In addition, the second predetermined parameter can be the number of reserved storage blocks determined in the process of writing stage test verification using the target model, or the number of reserved storage blocks determined by analyzing the output result of the output result of the target model in actual application. It should be noted that the above example of the first predetermined parameter and the second predetermined parameter is only an example, and the first predetermined parameter and the second predetermined parameter are not limited to the above example.

[0052] In an example embodiment, recovering the second storage block from the N pieces of the candidate storage blocks includes: sorting the N pieces of the candidate storage blocks in order of the number of times of reading from low to high; and recovering the second storage block from the N pieces of the candidate storage blocks in the order of sorting. In the embodiment, in addition to sorting the N pieces of the candidate storage blocks in order of the number of times of reading from low to high, the N pieces of the candidate storage blocks can also be sorted in order of priority from low to high, or in order of the proportion of effective data included in the N pieces of the candidate storage blocks from low to high, etc. It should be noted that the above example of the sorting method of the N pieces of the candidate storage blocks is only an example, and the sorting method of the N pieces of the candidate storage blocks is not limited to the above example.

[0053] In one example embodiment, in the case that the third number of the second storage blocks recovered in the above manner is less than the first difference, wherein the first difference is the difference between the first number and the second number, the method further comprises: determining the number N2 of the second candidate storage blocks to be screened according to the following formula:

[0054] N2>g*a2+ε

[0055] wherein the a2 is the difference between the first difference and the third number; determining the N2 second candidate storage blocks from the target hard disk according to the time stamp indicating data storage, and recovering a2 second storage blocks from the N2 second candidate storage blocks. In this embodiment, the N2 second candidate storage blocks can be determined from the target hard disk according to the time stamp of data storage from far to near, and the N2 second candidate storage blocks can be sorted according to the reading frequency of the N2 second candidate storage blocks from low to high, a2 second storage blocks can be recovered from the N2 second candidate storage blocks according to the sorting order, the N2 second candidate storage blocks can also be sorted according to the priority of the N2 second candidate storage blocks from low to high, a2 second storage blocks can be recovered from the N2 second candidate storage blocks according to the sorting order, the N2 second candidate storage blocks can also be sorted according to the proportion of valid data included in the N2 second candidate storage blocks from low to high, a2 second storage blocks can be recovered from the N2 second candidate storage blocks according to the sorting order, and so on. It should be noted that the above example of the sorting of the N2 second candidate blocks is only one example embodiment, and the sorting of the N2 second candidate blocks is not limited to the above example.

[0056] In an example embodiment, before the output result outputted by the target model after analyzing the received target data set is acquired, the method further comprises: acquiring the target data set obtained by the target monitoring device photographing the target object in the target region within the current predetermined time period; and inputting the target data set into the target model to instruct the target model to analyze the target data set. In this embodiment, there can be multiple target monitoring devices, and the target region can include multiple sub-regions. Thus, the multiple target monitoring devices can be used to photograph the target object in the multiple sub-regions, and the target data set obtained by the multiple target monitoring devices photographing the target object in the target region within the current predetermined time period can be acquired according to the priority of the multiple sub-regions or the multiple target monitoring devices. For example, when the target region has three sub-regions (sub-region 1, sub-region 2, and sub-region 3), and each sub-region has at least one target monitoring device, the priority of the three sub-regions is sub-region 1 > sub-region 2 > sub-region 3. The target data set obtained by the target monitoring device of the sub-region with higher priority photographing the target object in the sub-region within the current predetermined time period can be acquired in priority according to the priority order, and so on. When the target region has three sub-regions (sub-region 4, sub-region 5, and sub-region 6), and each sub-region has one target monitoring device (target monitoring device 1, target monitoring device 2, and target monitoring device 3), the priority of the three target monitoring devices is target monitoring device 1 > target monitoring device 2 > target monitoring device 3. The target data set obtained by the target monitoring device with higher priority photographing the target object in the corresponding sub-region within the current predetermined time period can be acquired in priority according to the priority order. It should be noted that the above examples of the target monitoring device and the target region are only example embodiments, and the target monitoring device and the target region are not limited to the above examples.

[0057] Obviously, the above-described embodiments are only part of the embodiments of the present application, not all the embodiments.

[0058] The present application will be described below in detail with reference to specific embodiments (taking the target model as a parking lot data flow model and the target region as a parking lot region as examples):

[0059] Step 1: receiving input data (corresponding to the above target data set) and performing step 2;

[0060] Step 2: pre-judging the input data by the parking lot data flow model (corresponding to the above target model), if the first stage is determined, performing step 3; if the second stage is determined, performing step 4; if the third stage is determined, performing step 8; if the fourth stage is determined, performing step 9;

[0061] Figure 3 This is a graph showing the data writing stage of a specific embodiment of the present invention, such as... Figure 3 As shown, the X-axis represents the frame data writing time, and the Y-axis represents the frame data volume. The frame data volumes corresponding to the curves between the horizontal dashed line X and the horizontal dashed line Y are the write data volumes for the second and fourth stages, respectively. According to the prediction criteria of the parking lot data flow model (for example, the disk will count the amount of data to be written within 1 second every 1 second (or 5 seconds, 30 seconds, 1 minute, etc.), that is, the parking lot data flow model provides judgment thresholds for each stage. When the write data volume reaches the threshold of a certain stage, that stage is determined as the current write stage), the entire data writing process is divided into 4 stages: Stage 1 (corresponding to the first stage mentioned above) is the low-speed stage, characterized by a small data write volume and low bandwidth resource requirements; Stage 2 ...3 (corresponding to the first stage mentioned above) is the low-speed stage, characterized by a small data write volume and low bandwidth resource requirements; Stage 4 (corresponding to the first stage mentioned above) is the low-speed stage, characterized by a small data write volume and low bandwidth resource requirements; Stage 5 (corresponding to the first stage mentioned above) is the low-speed stage, characterized by a small data write volume and low bandwidth resource requirements; Stage 6 (corresponding to the first stage mentioned above) is the low-speed stage, characterized by a small data write volume and low bandwidth resource requirements; Stage 7 (corresponding to the first stage mentioned above) is the low-speed stage, characterized by a small data write volume and low bandwidth resource requirements; Stage 8 (corresponding to the first stage mentioned above) is the low-speed stage, characterized by a small data write volume and low bandwidth resource requirements; Stage 9 (corresponding to the first stage mentioned above) is the low-speed stage, characterized by a small data write volume The second stage is the normal-speed stage, which serves as a transition from the low-speed to the high-speed stage. The data writing volume and bandwidth resource requirements are moderate, and it is generally sufficient to maintain a steady state of data. At the same time, the data writing volume of the next stage needs to be assessed and the corresponding available blocks need to be prepared in advance. The third stage (corresponding to the third stage above) is the high-speed stage, which is characterized by a large data writing volume and high bandwidth resource requirements. The fourth stage (corresponding to the fourth stage above) is roughly the same as the second stage (the trends of the data writing volume of the second and fourth stages are different. The trend of the data writing volume of the second stage is from less to more, while the trend of the data writing volume of the fourth stage is from more to less). The difference is that the fourth stage does not need to assess the amount of data to be written in the next stage or prepare the corresponding available blocks in advance.

[0062] Step 3: In the first stage, the TLC direct write scheme is adopted (corresponding to the multi-level storage cell direct write method mentioned above). Under the premise of ensuring bandwidth, the data GC migration from SLC (Single-Level Cell, which corresponds to the multi-level storage cell write method mentioned above) to TLC is performed at full speed (that is, the input data is first written to the SLC storage cell, and when the bandwidth is idle, the data stored in the SLC storage cell is written to the TLC storage cell), and then Step 2 is executed.

[0063] Step 4: The second stage adopts the TLC direct write scheme. Under the premise of achieving dynamic data balancing through flow control (i.e., ensuring normal storage of the written data in the second stage through flow control) and meeting the current bandwidth requirements, the number of blocks required for the third stage (corresponding to the first number mentioned above) is evaluated, and step 5 is executed.

[0064] Step 5: Compare the number of remaining blocks (corresponding to the second number mentioned above) with the number of blocks required in the third stage. If the number of remaining blocks is greater than the required number of blocks, proceed to step 2; otherwise, proceed to step 6.

[0065] Step 6: Forced garbage collection is performed under the requirement of ensuring the bandwidth of the second stage, and the sum of the number of remaining blocks (corresponding to the third number) after garbage collection of blocks with a useful information (corresponding to the valid data) proportion less than 1 / 5 (corresponding to the predetermined threshold) and the number of existing remaining blocks (corresponding to the second number) is compared with the number of blocks required in the third stage. If it is greater, step 2 is performed; otherwise, step 7 is performed.

[0066] Step 7: The available block recycling method suitable for the parking lot (corresponding to the above-mentioned method two) is used to complete the recycling of the difference block (corresponding to the above-mentioned N2), and step 2 is performed.

[0067] Step 8: The third stage adopts the SLC writing scheme, directly writes data to the pre-prepared data block (corresponding to the target available block), and does not open the GC function during the period to avoid the limitation of bandwidth resources leading to the reduction of storage performance; and step 2 is performed.

[0068] Step 9: The fourth stage adopts the TLC direct writing scheme, opens the GC process, adjusts the flow control to complete the dynamic balance of data, and performs step 2.

[0069] The embodiment of the application also provides a available block recycling process, Figure 4 is a flowchart of the available block recycling according to the embodiment of the application, as Figure 4 shown, the process includes the following steps:

[0070] S402, the available block recycling process starts;

[0071] S404, according to the metadata timestamp in the data saving, the old data block is screened, and the screening standard is as follows:

[0072] N>g*a+ε

[0073] Wherein, a represents the expected number of available blocks (corresponding to the target available block), which is obtained by dividing the task amount Y (corresponding to the predicted data amount) by the capacity of a single storage block; g is the amplification factor (corresponding to the first predetermined parameter), which can be defined as 1.5 (and can also be defined as 1.3, 1.4, 1.6, etc.); ε is a fine-tuning parameter (corresponding to the second predetermined parameter), which is obtained by training the parking lot data flow model, and is generally between 3-10.

[0074] S406, the N data blocks (corresponding to the N first candidate storage blocks) are sorted in ascending order of the read frequency, wherein the read frequency of the data block is maintained by a system block information table, and the system block information table stores all information of the storage block, such as the read-write frequency, the erase frequency, and the storage block writing mode.

[0075] S408, the first a data blocks are recycled by GC;

[0076] S410, the recycled blocks are placed in the available blocks for large data service;

[0077] S412, the available block recycling process is ended.

[0078] As can be seen from the foregoing embodiments, the current writing stage X of the writing service is evaluated based on the data amount of the current writing service by using the parking lot data flow model n At the same time, whether the next stage is a large service amount stage and the corresponding task amount Y are determined, the bandwidth resource and the storage resource of the solid state disk are dynamically adjusted according to the service stage and the data amount to be written, in order to ensure that the solid state disk has sufficient storage space to save large capacity data, through the available block recycling method, based on the time stamp carried in the metadata in the stored data in the solid state disk, the earliest data is found through the time stamp when recycling the available block, and the read frequency of the earliest data is combined as a judgment standard, the data written earliest and read least is recycled preferentially, so as to achieve the purpose of minimizing the influence on the original video data.

[0079] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, or network device, etc.) execute the method described in each embodiment of the present application.

[0080] In this embodiment, a data writing device is also provided, which is used to implement the above embodiments and preferred embodiments, and has been described above. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware is also possible and is contemplated.

[0081] Figure 5 is a structural block diagram of a data writing device according to an embodiment of the present application, as shown, the device comprises: Figure 5

[0082] A first acquisition module 52 is configured to acquire an output result output by a target model after analyzing a target data set, wherein the target data set is data received in a current predetermined time period and to be written into a target hard disk, and the output result comprises a current writing stage for indicating a current writing of data into the target hard disk;

[0083] A first determination module 54 is configured to determine a target writing mode of the target data set based on the current writing stage;

[0084] A writing module 56 is configured to write the target data set into the target hard disk according to the target writing mode.

[0085] In an example embodiment, the target model is configured to analyze the target data set and output the output result by the following manner:

[0086] In a case where a data amount of the target data set is less than or equal to a first data amount threshold, it is determined that the current writing stage included in the output result is a first stage;

[0087] In a case where the data amount of the target data set is greater than the first data amount threshold and less than or equal to a second data amount threshold, it is determined that the current writing stage included in the output result is one of a second stage and a fourth stage;

[0088] In a case where the data amount of the target data set is greater than the second data amount threshold, it is determined that the current writing stage included in the output result is a third stage.

[0089] In an example embodiment, the first determination module 54 comprises:

[0090] A first determination sub-module is configured to, in a case where the current writing stage is the first stage, the second stage or the fourth stage, determine that the target writing mode is a multi-layer storage unit direct writing mode;

[0091] A second determination sub-module is configured to, in a case where the current writing stage is the third stage, determine that the target writing mode is a single-layer storage unit writing mode.

[0092] ​In an example embodiment, the target model is further configured to compare the data amount of the target data set with a data amount of a first data set in a case where it is determined that the data amount of the target data set is greater than a first data amount threshold and less than or equal to a second data amount threshold; determine that the current write stage is the second stage in a case where it is determined that the data amount of the target data set is greater than the data amount of the first data set; and determine that the current write stage is the fourth stage in a case where it is determined that the data amount of the target data set is less than the data amount of the first data set.

[0093] The first data set is a data set received by the target model before the target data set is received.

[0094] In an example embodiment, in a case where the current write stage included in the output result is the second stage, the output result further includes a predicted data amount of data to be written into the target hard disk in the third stage, and the apparatus further includes:

[0095] A second determining module configured to determine, based on the predicted data amount, a first number of available blocks needed by the target hard disk to store the predicted data amount in the third stage, wherein the available blocks are first storage blocks included in the target hard disk and not occupied by other data.

[0096] A third determining module configured to determine a second number of the available blocks included in the target hard disk before entering the third stage.

[0097] A fourth determining module configured to determine, based on the first number and the second number, target available blocks needed by the target hard disk to store the predicted data amount in the third stage.

[0098] In an example embodiment, the fourth determining module further includes:

[0099] A comparing sub-module configured to compare the first number with the second number.

[0100] A recycling sub-module configured to recycle, in a case where it is determined that the first number is greater than the second number, second storage blocks included in the target hard disk and occupied by the other data according to a target recycling manner, and determine the available blocks of the second number and the recycled second storage blocks as the target available blocks.

[0101] A third determining sub-module configured to, in a case where it is determined that the first number is less than or equal to the second number, determine the available blocks of the first number from the available blocks of the second number, and determine the available blocks of the first number as the target available blocks.

[0102] In an example embodiment, the recycling submodule is further configured to, in a case where the first quantity is greater than the second quantity, recycle the second storage blocks included in the target hard disk and occupied by other data in at least one of the following manners:

[0103] Manner one: recycle the second storage blocks included in the target hard disk and occupied by the other data, in which the valid data of the other data accounts for less than a predetermined threshold;

[0104] Manner two: determine the quantity N1 of the first candidate storage blocks to be screened according to the following formula:

[0105] N1>g*a1+ε

[0106] wherein g is a first predetermined parameter, ε is a second predetermined parameter, a1 is the first quantity, and a1 is the predicted data quantity divided by the capacity of a single storage block included in the target hard disk;

[0107] determine N first candidate storage blocks from the target hard disk according to the timestamps indicating data storage, and recycle the second storage blocks from the N first candidate storage blocks.

[0108] In an example embodiment, the recycling submodule is further configured to recycle the second storage blocks from the N candidate storage blocks by the following operations:

[0109] sort the N candidate storage blocks in the order of the number of times of reading from low to high;

[0110] recycle the second storage blocks from the N candidate storage blocks according to the sorting order.

[0111] In an example embodiment, the apparatus further comprises:

[0112] a fifth determining module configured to, in a case where a third quantity of the second storage blocks recycled according to the manner one is less than a first difference, wherein the first difference is the difference between the first quantity and the second quantity, determine the quantity N2 of the second candidate storage blocks to be screened according to the following formula:

[0113] N2>g*a2+ε

[0114] wherein a2 is the difference between the first difference and the third quantity;

[0115] a sixth determining module configured to determine N2 second candidate storage blocks from the target hard disk according to the timestamps indicating data storage, and recycle a2 second storage blocks from the N2 second candidate storage blocks.

[0116] In an example embodiment, the apparatus further comprises:

[0117] a second obtaining module, configured to obtain the target data set obtained by the target monitoring device in the current predetermined time period for photographing the target object in the target area;

[0118] an input module, configured to input the target data set into the target model to instruct the target model to analyze the target data set.

[0119] It should be noted that each of the above modules can be implemented by software or hardware, and for the latter, the following implementation manners can be used, but are not limited thereto: all of the above modules are located in the same processor; or each of the above modules is located in a different processor in any combination.

[0120] Embodiments of the present application also provide a computer readable storage medium, which stores a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0121] In an example embodiment, the computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0122] Embodiments of the present application also provide an electronic device, which comprises a memory storing a computer program and a processor configured to execute the computer program to perform the steps in any of the above method embodiments.

[0123] In an example embodiment, the electronic device can further comprise a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0124] The specific examples in the present embodiment can refer to the examples described in the above embodiments and example implementations, and the present embodiment will not be described here again.

[0125] The data writing method provided by the application dynamically adjusts bandwidth resources and writing modes (i.e. dynamic adjustment of dynamic SLC writing strategy and TLC direct writing strategy) according to current stage data traffic and future data traffic, reasonably completes scheduling of stored data, and improves reliability and scheduling efficiency of firmware, i.e. the current writing stage of writing data is evaluated by using a target model, and bandwidth resources and storage resources of a target hard disk are dynamically adjusted based on the current writing stage, so that fast response can be obtained during large data traffic, and writing efficiency of data is ensured, meanwhile, GC tasks are dispatched in different writing stages, the utilization rate of bandwidth resources is improved while the writing efficiency and reliability of data are ensured, the problem that solid state disks enter low water level quickly due to large bandwidth pressure in the monitoring field is further solved, and the reliability and scheduling efficiency of firmware data scheduling are ensured.

[0126] Obviously, those skilled in the art should understand that the modules or steps of the application described above can be realized by general computing devices, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices, and can be realized by program codes executable by the computing devices, so that they can be stored in storage devices and executed by the computing devices, and in some cases, the steps shown or described can be executed in different order, or they can be manufactured into individual integrated circuit modules, or multiple modules or steps can be manufactured into a single integrated circuit module. Thus, the application is not limited to any specific combination of hardware and software.

[0127] The above only describes the preferred embodiments of the application and is not used to limit the application. For those skilled in the art, the application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the principles of the application should be included in the protection scope of the application.

Claims

1. A data write method, characterized by, The method comprises: obtaining an output result output by a target model after analyzing a target data set received by the target model, wherein the target data set is data received in a current predetermined time period and to be written into a target hard disk, the output result includes a current writing stage indicating that data is currently being written into the target hard disk, and the output result is determined by the target model based on a data amount of the target data set; determining a target writing mode of the target data set based on the current writing stage; writing the target data set into the target hard disk in the target writing mode; the target model is configured to analyze the target data set and output the output result by: determining that the current writing stage included in the output result is a first stage when it is determined that the data amount of the target data set is less than or equal to a first data amount threshold; determining that the current writing stage included in the output result is one of a second stage and a fourth stage when it is determined that the data amount of the target data set is greater than the first data amount threshold and less than or equal to a second data amount threshold; and determining that the current writing stage included in the output result is a third stage when it is determined that the data amount of the target data set is greater than the second data amount threshold, wherein the target model is further configured to predict a predicted data amount of data to be written into the target hard disk in the third stage, and to schedule storage resources in the target hard disk based on the predicted data amount; determining the target writing mode of the target data set based on the current writing stage comprises: determining that the target writing mode is a multi-layer storage unit direct writing mode when the current writing stage is the first stage, the second stage, or the fourth stage; and determining that the target writing mode is a single-layer storage unit writing mode when the current writing stage is the third stage.

2. The method of claim 1, wherein, The target model is further configured to compare the data amount of the target data set with a data amount of a first data set when it is determined that the data amount of the target data set is greater than the first data amount threshold and less than or equal to the second data amount threshold. determining that the current writing stage is the second stage when it is determined that the data amount of the target data set is greater than the data amount of the first data set, and determining that the current writing stage is the fourth stage when it is determined that the data amount of the target data set is less than the data amount of the first data set; wherein the first data set is a data set received by the target model before the target data set is received.

3. The method of claim 2, wherein, when the current writing stage included in the output result is the second stage, the output result further includes a predicted data amount of data to be written into the target hard disk in the third stage, and the method further comprises: determining a first number of available blocks needed by the target hard disk to store the predicted data amount in the third stage based on the predicted data amount, wherein the available blocks are first storage blocks included in the target hard disk and not occupied by other data. determining a second number of the available blocks included in the target hard disk before entering the third stage; determining target available blocks needed by the target hard disk to store the predicted data amount in the third stage based on the first number and the second number.

4. The method of claim 3, wherein, determining target available blocks needed by the target hard disk to store the predicted data amount in the third stage based on the first number and the second number includes: comparing the first number with the second number; in a case where it is determined that the first number is greater than the second number, recycling second storage blocks included in the target hard disk and occupied by the other data in a target recycling manner, and determining the second number of the available blocks and the recycled second storage blocks as the target available blocks; in a case where it is determined that the first number is less than or equal to the second number, determining the first number of the available blocks from the second number of the available blocks, and determining the first number of the available blocks as the target available blocks.

5. The method of claim 4, wherein, in a case where it is determined that the first number is greater than the second number, recycling second storage blocks included in the target hard disk and occupied by the other data in a target recycling manner includes at least one of the following manners: Manner one: recycling the second storage blocks in which the proportion of valid data in the other data included in the target hard disk is less than a predetermined threshold; Manner two: determining a number N1 of first candidate storage blocks to be screened according to the following formula: ， wherein the is a first predetermined parameter, and the is a second predetermined parameter, and the 1 is the first number, and the 1 is the predicted data amount divided by the capacity of a single storage block included in the target hard disk. determining from the target hard disk a timestamp indicative of data storage de-allocating the first candidate storage block, and reclaiming the second storage block from the first candidate storage block.

6. The method of claim 5, wherein, From reclaiming the second storage block from the first candidate storage block includes: According to The first candidate storage blocks are sorted in order of the number of times each block is read from low to high The first candidate storage blocks are sorted; reclaiming the second storage block from the first candidate storage block in the order. reclaiming the second storage block from the first candidate storage block in the order.​ 7. The method of claim 5, wherein, in a case where a third number of the second storage blocks recycled according to the manner one is less than a first difference value, wherein the first difference value is a difference between the first number and the second number, the method further includes: determining a number N2 of second candidate storage blocks to be screened according to the following formula: ， wherein the is a difference between the first difference and the third quantity; N2 blocks of the second candidate storage blocks are determined from the target hard disk according to timestamps used to indicate data storage, and the N2 blocks of the second candidate storage blocks are recovered blocks of the second storage blocks.

8. The method of claim 1, wherein, before obtaining an output result output by a target model after analyzing a target data set received by the target model, the method further includes: obtaining the target data set obtained by a target monitoring device in photographing a target object in a target region in the current predetermined time period; inputting the target data set into the target model to instruct the target model to analyze the target data set.

9. A data writing apparatus, characterized by comprising: includes: a first obtaining module, configured to obtain an output result output by a target model after analyzing a target data set received by the target model, wherein the target data set is data received in a current predetermined time period and to be written into a target hard disk, and the output result includes a current writing stage for indicating that data is currently written into the target hard disk, and the output result is determined by the target model based on a data amount of the target data set; a first determining module, configured to determine a target writing manner of the target data set based on the current writing stage; a writing module, configured to write the target data set into the target hard disk in the target writing manner; The target model is configured to analyze the target data set and output the output result by: determining the current writing stage included in the output result as a first stage when determining that a data amount of the target data set is less than or equal to a first data amount threshold; determining the current writing stage included in the output result as one of a second stage and a fourth stage when determining that the data amount of the target data set is greater than the first data amount threshold and less than or equal to a second data amount threshold; and determining the current writing stage included in the output result as a third stage when determining that the data amount of the target data set is greater than the second data amount threshold; and predicting a predicted data amount of data to be written into the target hard disk in the third stage, wherein a storage resource in the target hard disk is scheduled based on the predicted data amount. The device is further configured to: determine the target writing mode as a multi-layer storage unit direct writing mode when the current writing stage is the first stage, the second stage, or the fourth stage; and determine the target writing mode as a single-layer storage unit writing mode when the current writing stage is the third stage.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, wherein the computer program is executed by a processor to implement the steps of the method in any one of claims 1 to 8.

11. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 8.

Images