Method, apparatus and system for processing data

Abstract

The application discloses a data processing method, device and system. The method comprises the following steps: obtaining training samples composed of at least two different types of image samples, wherein the image density displayed in the different types of image samples is in different density interval values; training a first density estimation model by using the different types of image samples, and obtaining density distribution results of the different types of image samples, wherein the different types of image samples are classified and marked in the training process. The application solves the technical problem that the estimation result of the density estimation model is inaccurate when the density estimation model estimates the image in the related art.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, in particular to a data processing method, device and system. BACKGROUND

[0002] At present, a large amount of sample data needs to be used to train the density estimation model, but the data used to train the density estimation model is very limited, the sample data scale is generally small, and the efficiency of labeling the data is low. Therefore, training the density estimation model by using such sample data will result in low accuracy of the density estimation model, and further result in inaccurate estimation results of the density estimation model in estimating the image.

[0003] At present, no effective solution has been proposed for the above problems. SUMMARY

[0004] The embodiments of the present application provide a data processing method, device and system to at least solve the technical problem of inaccurate estimation results of the density estimation model in estimating the image.

[0005] According to an aspect of the embodiments of the present application, a data processing method is provided, characterized in that it comprises: obtaining a training sample composed of at least two different types of image samples, wherein the image density displayed in the different types of image samples is in different density interval values, and the image density of the image sample represents the display density of the target object in the image sample; training a first density estimation model using different types of image samples, and obtaining the density distribution result of different types of image samples, wherein the different types of image samples are classified and labeled in the training process.

[0006] According to another aspect of the embodiments of the present application, a data processing method is also provided, characterized in that it comprises: obtaining a video to be recognized, wherein the video comprises multiple frames of images; analyzing the images in the video using the first density estimation model to identify the density of the target object displayed in each frame of image; wherein the first density estimation model is trained using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0007] According to another aspect of the embodiments of the present application, a data processing method is also provided, which comprises: obtaining a video in a predetermined space within a predetermined time period, wherein the video contains products in the predetermined space within the predetermined time period, and the video comprises multiple images; analyzing the images in the video by using a first density estimation model to identify the display density of the products in each image; and outputting prompt information based on the display density of the products in the images, wherein the prompt information comprises at least one of the following: product occlusion, unreasonable product display quantity, and product type error; wherein different types of image samples are used to train the first density estimation model, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0008] According to another aspect of the embodiments of the present application, a data processing method is also provided, which comprises: a cloud server receiving an identification message from a client, wherein the identification message carries identification information representing a video to be identified; the cloud server obtaining the video to be identified based on the identification information; the cloud server analyzing the images in the video by using a first density estimation model to identify the density of target objects displayed in each image; and the cloud server returning the identification result to the client; wherein different types of image samples are used to train the first density estimation model, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0009] According to another aspect of the embodiments of the present application, a data processing method is also provided, which comprises: displaying a captured vehicle video in a road traffic video interface, wherein each image in the vehicle video contains different types of vehicles; analyzing any one image in the vehicle video by using a first density estimation model to identify the density of the vehicles displayed in each image; and controlling the road traffic light to output corresponding control information based on the density of the vehicles displayed in the images; wherein different types of image samples are used to train the first density estimation model, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0010] According to another aspect of the embodiments of the present application, a data processing method is also provided, comprising: displaying a video to be recognized in an interactive interface, wherein the interactive interface provides at least one space for interaction, and the video comprises multiple images; triggering a control generation instruction, and calling a first density estimation model in response to the instruction; analyzing the images in the video by using the first density estimation model to recognize the density of a target object displayed in each image; and displaying the density of the target object displayed in each image in a display interface; wherein the first density estimation model is trained by using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0011] According to another aspect of the embodiments of the present application, a data processing apparatus is also provided, comprising: an acquisition module configured to acquire training samples composed of at least two different types of image samples, wherein the image density displayed in the different types of image samples is in different density interval values, and the image density of the image samples represents the display density of a target object in the image samples; and a training module configured to train a first density estimation model by using the different types of image samples, and acquire the density distribution results of the different types of image samples, wherein the different types of image samples are classified and labeled in the training process.

[0012] According to another aspect of the embodiments of the present application, a data processing apparatus is also provided, comprising: an acquisition module configured to acquire a video to be recognized, wherein the video comprises multiple images; and an analysis module configured to analyze the images in the video by using a first density estimation model to recognize the density of a target object displayed in each image, wherein the first density estimation model is trained by using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0013] According to another aspect of the embodiments of the present application, a data processing apparatus is also provided, comprising: an acquisition module configured to acquire a video in a predetermined space within a predetermined time period, wherein the video contains products located in the predetermined space within the predetermined time period, and the video comprises multiple images; and an analysis module configured to analyze the images in the video by using a first density estimation model to recognize the display density of the products displayed in each image; and outputting prompt information based on the display density of the products displayed in the images, wherein the prompt information comprises at least one of the following: product occlusion, unreasonable product display quantity, and product type error; wherein the first density estimation model is trained by using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0014] According to another aspect of the embodiments of the present application, a data processing apparatus is also provided, comprising: a display module configured to display a live video to be recognized in a live interface, wherein different types of products are played in the live video; a response module configured to respond to a detection instruction sensed in the live interface, and invoke a first density estimation model; and a display module configured to display the density of products in each frame of image in the live video in the live interface, wherein the first density estimation model is used to analyze the image in the video, and the density of the target object displayed in each frame of image is recognized; wherein the first density estimation model is trained by using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0015] According to another aspect of the embodiments of the present application, a data processing apparatus is also provided, comprising: a receiving module configured to receive a recognition message from a client, wherein the recognition message carries identification information representing a video to be recognized; an obtaining module configured to obtain the video to be recognized based on the identification information; an analysis module configured to analyze the image in the video by using a first density estimation model, and recognize the density of the target object displayed in each frame of image; and a returning module configured to return the recognition result to the client; wherein the first density estimation model is trained by using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0016] According to another aspect of the embodiments of the present application, a data processing apparatus is also provided, comprising: a display module configured to display a captured vehicle video in a road traffic video interface, wherein different types of vehicles are displayed in each frame of image included in the vehicle video; an identification module configured to analyze any frame of image in the vehicle video by using a first density estimation model, and recognize the density of the vehicle displayed in each frame of image; and a control module configured to control a road traffic light to output corresponding control information based on the density of the vehicle displayed in the image; wherein the first density estimation model is trained by using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0017] According to a further aspect of the embodiments of the present application, a data processing apparatus is also provided, comprising: a display module configured to display a video to be recognized in an interactive interface, wherein the interactive interface provides at least one control for interaction, and the video comprises a plurality of images; a triggering module configured to trigger the control to generate an instruction, and invoke a first density estimation model in response to the instruction; a recognition module configured to analyze the images in the video using the first density estimation model, and recognize the density of a target object displayed in each image; and a display module configured to display the density of the target object displayed in each image in the display interface; wherein the first density estimation model is trained using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled during the training process.

[0018] In the embodiments of the present application, first, training samples composed of at least two different types of image samples are obtained, wherein the image density displayed in the different types of image samples is in different density interval values, and the image density of the image samples represents the display density of the target object in the image samples; the first density estimation model is trained using the different types of image samples, and the density distribution results of the different types of image samples are obtained, wherein the different types of image samples are classified and labeled during the training process, thereby achieving more accurate training of the first density estimation model. It is easy to note that the display density of the target object in the different types of image samples is different, and therefore, training the first density estimation model using the different types of image samples can improve the accuracy of the density estimation model when estimating images, and meanwhile, classifying and labeling the different types of image samples during the training process can enable the first density estimation model to learn the different types of image samples and output the density distribution results of the different types of image samples, thereby solving the technical problem of inaccurate estimation results of the density estimation model when estimating images in the related art. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings, which are included to provide a further understanding of the present application, form a part of the present application and illustrate the illustrative embodiments of the present application and together with the description given below, serve to explain the present application, and do not limit the present application in any way. In the drawings:

[0020] Figure 1 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing an image processing method according to an embodiment of the present application;

[0021] Figure 2 is a flowchart of a data processing method according to an embodiment 1 of the present application;

[0022] Figure 3 is a schematic diagram of an interactive interface according to an embodiment 1 of the present application;

[0023] Figure 4 is a data processing structure block diagram according to Embodiment 1 of the present application;

[0024] Figure 5 is a data processing structure block diagram according to Embodiment 1 of the present application;

[0025] Figure 6 is a data processing structure block diagram according to Embodiment 1 of the present application;

[0026] Figure 7 is a data processing method flow chart according to Embodiment 1 of the present application;

[0027] Figure 8 is a data processing method flow chart according to Embodiment 2 of the present application;

[0028] Figure 9 is a data processing method flow chart according to Embodiment 3 of the present application;

[0029] Figure 10 is a data processing method flow chart according to Embodiment 4 of the present application;

[0030] Figure 11 is a data processing method flow chart according to Embodiment 5 of the present application;

[0031] Figure 12 is a data processing method flow chart according to Embodiment 6 of the present application;

[0032] Figure 13 is a data processing device schematic diagram according to Embodiment 7 of the present application;

[0033] Figure 14 is a data processing device schematic diagram according to Embodiment 8 of the present application;

[0034] Figure 15 is a data processing device schematic diagram according to Embodiment 9 of the present application;

[0035] Figure 16 is a data processing device schematic diagram according to Embodiment 10 of the present application;

[0036] Figure 17 is a data processing device schematic diagram according to Embodiment 11 of the present application;

[0037] Figure 18 is a data processing device schematic diagram according to Embodiment 12 of the present application;

[0038] Figure 19is a flow chart of a data processing method according to the embodiment 13 of the present application;

[0039] Figure 20 is a schematic diagram of a data processing device according to the embodiment 14 of the present application;

[0040] Figure 21 is a structural block diagram of a computer terminal according to the embodiment 15 of the present application. DETAILED DESCRIPTION

[0041] In order to make the personnel in the technical field better understand the scheme of the present application, the technical scheme in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the person of ordinary skill in the art without creative labor should belong to the protection scope of the present application.

[0042] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0043] Embodiment 1

[0044] According to the embodiments of the present application, a data processing method embodiment is also provided. It should be noted that the steps shown in the flow chart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flow chart, in some cases, the steps shown or described herein can be executed in an order different from that herein.

[0045] The method embodiments provided by the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 A hardware structural block diagram of a computer terminal (or mobile device) for implementing the data processing method is shown. As shown in the figure, Figure 1As shown, the computer terminal 10 (or mobile device 10) can include one or more processors 102 (the processor 102 can include, but not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. Those skilled in the art can understand that Figure 1 The structure shown is only schematic, which does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 can include more or fewer components than those shown in Figure 1 or have a different configuration than that shown in Figure 1 .

[0046] It should be noted that the one or more processors 102 and / or other data processing circuits described above can be generally referred to herein as "data processing circuits". The data processing circuits can be embodied in whole or in part as software, hardware, firmware or any other combination. In addition, the data processing circuits can be a single independent processing module, or any one of the other elements incorporated into the computer terminal 10 (or mobile device) in whole or in part. The data processing circuits serve as a processor to control, for example, the selection of the variable resistance terminal path connected to the interface.

[0047] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage means corresponding to the data processing method in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, i.e. implements the data processing method described above. The memory 104 can include a high-speed random access memory, and can also include a non-volatile memory such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 can further include a memory remotely located with respect to the processor 102, which can be connected to the computer terminal 10 through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0048] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0049] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0050] It should be noted here that, in some optional embodiments, the above... Figure 1 The computer device (or mobile device) shown may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that... Figure 1 This is only one instance of a specific particular instance and is intended to illustrate the types of components that may exist in the aforementioned computer device (or mobile device).

[0051] Under the aforementioned operating environment, this application provides the following: Figure 2 The data processing method shown. Figure 2 This is a flowchart of a data processing method according to an embodiment of this application. For example... Figure 2 As shown, the method includes the following steps:

[0052] Step S202: Obtain training samples consisting of at least two different types of image samples.

[0053] Among them, the image density displayed in different types of image samples is in different density ranges, and the image density of the image sample represents the display density of the target object in the image sample.

[0054] The image samples in the above steps can be images of crowds in a square or football field, with the target object being people; image samples can also be images of attendees in a meeting, with the target object being attendees; image samples can also be video frames about products in a live broadcast, with the target object being products; image samples can also be vehicle images in a traffic scene, with the target object being vehicles; image samples can also be animal images in a zoo scene, with the target object being animals.

[0055] The density interval value in the above step can be freely set. When the display density of the target object in the image is larger, the density interval value is larger; and when the display density of the target object in the image is smaller, the density interval value is smaller.

[0056] In an optional embodiment, two different types of image samples can be combined to form a training sample. For example, the two different types of samples can be combined by left-right splicing, or by up-down splicing, or by superimposing.

[0057] In step S204, the first density estimation model is trained by using different types of image samples, and a density distribution result of the different types of image samples is obtained.

[0058] In the training process, the different types of image samples are classified and labeled.

[0059] The first density estimation model in the above step can be a convolutional neural network model, which is used to identify the density in the image, so as to output the density distribution result of the different types of image samples.

[0060] In an optional embodiment, the first density estimation model can be trained by using the first type of image sample first, and then trained by using the second type of image sample. In this way, the density estimation model can be more accurately trained, so as to improve the accuracy of the density estimation model.

[0061] In an optional embodiment, in the density distribution result, the target object in the different types of image samples can be represented by points. When the target object is in a larger image density, the points formed in the density distribution result are relatively dense; and when the target object is in a smaller image density, the points formed in the density distribution result are relatively sparse, so as to clearly indicate the display density of the target object in the image sample. Further, the density distribution result of the target object in the different types of image samples can be displayed in one graph, so that the user can more intuitively feel the density difference of the target object in the different types of image samples.

[0062] In another optional embodiment, in the density distribution result, the density of the target object in the different types of image samples can be displayed by a pie-shaped body. In a plurality of types of image samples, when the target object in an image sample is in a larger image density, the percentage of the pie-shaped graph occupied by the target object is larger; and when the target object in an image sample is in a smaller image density, the percentage of the pie-shaped graph occupied by the target object is smaller, so that the user can clearly know the display density distribution of the target object in the different types of image samples. Similarly, the density distribution result can also be displayed by a column chart or a line chart.

[0063] In another optional embodiment, in the density distribution result, the density of the target object in different types of image samples can be divided into at least two levels, for example, a first level and a second level, where the first level represents that the density of the target object in the image sample is relatively large, and the second level represents that the density of the target object in the image sample is relatively small. In the density distribution result, the density level of the target object in the first type of image sample can be represented in the form of text as the first level, and the density level of the target object in the second type of image sample can be represented in the form of text as the second level, so as to enable the user to view in the form of text.

[0064] The classification and labeling in the above steps are used to set labels for the image samples, so as to distinguish different types of image samples, that is, distinguish image samples with different densities.

[0065] In another optional embodiment, in the training process, different types of image samples can be classified and labeled by numbers, where the labeling can also be referred to as setting labels. For example, when the density of the target object in the image sample is relatively large, the label thereof can be set as 0; when the density of the target object in the image sample is medium, the label thereof can be set as 1; and when the density of the target object in the image sample is relatively small, the label thereof can be set as 2. By labeling the image samples, the first density estimation model can identify the range of the density of the target object in the image when identifying the image. Different types of image samples can also be classified and labeled by text. For example, when the density of the target object in the image sample is relatively large, the label thereof can be set as density large; when the density of the target object in the image sample is medium, the label thereof can be set as density medium; and when the density of the target object in the image sample is relatively small, the label thereof can be set as density small.

[0066] In another optional embodiment, in order to better train the first density estimation model, at least two training samples composed of different types of image samples can be transmitted to a corresponding processing device for processing, for example, directly transmitted to a computer terminal (for example, a notebook computer, a personal computer, etc.) of the user for processing, or transmitted to a cloud server through the computer terminal of the user for processing. It should be noted that, since a large amount of computing resources are required to train the first density estimation model, the processing device in the embodiments of the present application is taken as the cloud server for illustration.

[0067] For example, in order to facilitate the user to upload different types of image samples for training, an interactive interface can be provided for the user, such as Figure 3As shown, the user can select the image samples to be uploaded from the training samples one by one by clicking the "Select Image Sample" button, or select multiple image samples in batches, and upload the image samples to the cloud server to train the first density estimation model by clicking the "Upload" button. In addition, to facilitate the user to confirm whether the uploaded image samples are the image samples required to train the first density estimation model, the generated image samples can be displayed in the "Image Sample Display" area, and after the user confirms that there is no error, the data upload is performed by clicking the "Upload" button.

[0068] The first density estimation model trained through the above steps can be used to estimate the density of the target object in the image. For example, the first density estimation model can identify the density of the target object in the image. The following examples are used for illustration.

[0069] In another optional embodiment, taking a football field as an example, the image of the football field can be input into the first density estimation model. The first density estimation model can determine whether the density of the spectators in the football field is large according to the image of the football field. If it is determined that the density of the spectators is large, the management personnel needs to be reminded to strengthen the security measures. If it is determined that the density of the spectators is small, the management personnel does not need to be reminded to strengthen the security measures.

[0070] In another optional embodiment, taking a live scene as an example, the video frame of the commodity in the live scene can be input into the first density estimation model. The first density estimation model can remind the user to reposition the commodity according to whether the density of the commodity in the video frame is too large. If it is determined that the density of the commodity is too large, it is considered that this kind of arrangement may affect the sales of the commodity, and the user can be reminded to reposition the commodity. If it is determined that the density of the commodity is small, it is considered that this kind of arrangement will not affect the sales of the commodity, and the user does not need to be reminded to reposition the commodity.

[0071] In yet another optional embodiment, taking a traffic scene as an example, the image of the traffic scene can be input into the first density estimation model. The first density estimation model can determine the vehicle density in the traffic scene according to the image of the traffic scene. If the vehicle density is too large, the management personnel can be reminded to dredge the traffic of the road section. If the vehicle density is too small, the management personnel does not need to be reminded to dredge the traffic of the road section.

[0072] By the scheme provided in the above embodiments of the present application, first, training samples composed of at least two different types of image samples are acquired, wherein the image density displayed in the different types of image samples is in different density interval values, and the image density of the image sample represents the display density of the target object in the image sample; the first density estimation model is trained using the different types of image samples, and the density distribution results of the different types of image samples are acquired, wherein in the training process, the different types of image samples are classified and labeled, thereby achieving more accurate training of the first density estimation model. It is easy to note that the display density of the target object in the different types of image samples is not the same, and therefore, training the first density estimation model using the different types of image samples can improve the accuracy of the density estimation model when estimating the image, and meanwhile, classifying and labeling the different types of image samples in the training process can enable the first density estimation model to learn the different types of image samples and output the density distribution results of the different types of image samples, thereby solving the technical problem of inaccurate estimation results of the density estimation model when estimating the image in the related art.

[0073] In the above embodiments of the present application, the training samples composed of at least two different types of image samples are acquired: an image sample set is acquired; a plurality of image dimension parameters are acquired, and the plurality of image dimension parameters are combined to generate a classification condition, wherein the image dimension parameters include at least one of the following: image background, image scene, density value of a target object displayed in the image, image view angle, image shooting distance, occlusion degree, and image definition; and the image sample set is classified based on the classification condition to obtain at least two different types of image samples.

[0074] In an optional embodiment, an image sample set including a plurality of image samples can be acquired, wherein the image samples in the image sample set can be images under different scenes, can be images of different densities, and can be images shot under different view angles.

[0075] The image dimension parameters in the above steps can be acquired from the attributes of the image samples, can be acquired from the labels in the image samples, and can be acquired from the display screens in the image samples.

[0076] The image background in the above steps can be divided into solid color background, patterned background, etc.; the image scene can be divided into football scene, conference scene, traffic scene, etc.; the density value of the target object displayed in the image can be divided into the density of the audience in the football scene, the density of the participants in the conference scene, the density of the vehicles in the traffic scene, etc.; the image view angle can be divided into wide-angle shooting, normal shooting, etc.; the image shooting distance can be divided into long shot, normal distance shooting, close-up shooting, etc.; the occlusion degree can be divided into no occlusion, partial occlusion, and full occlusion, etc.; and the image definition can be divided into blur, normal, and clear, etc.

[0077] In an optional embodiment, a plurality of image dimension parameters of each image sample in the image sample set can be acquired, and the plurality of image dimension parameters can be freely combined to generate the classification condition.

[0078] For example, the image samples in the image sample set can be classified according to the image scene and the density value of the target object actually present in the image, and the first density estimation model can be trained by the two types of classified image samples, so as to improve the recognition accuracy of the first density estimation model for the two types of images.

[0079] In the above embodiments of the present application, the first density estimation model is trained by using different types of image samples, and the density distribution results of different types of image samples are acquired, including: extracting sample features of different types of image samples to obtain a sample feature set; inputting the sample feature set into an encoder in the first density estimation model to output encoded feature data; using a classifier to perform constraint processing on the encoded feature data, wherein the constraint processing represents classification and labeling of the encoded feature data; inputting the constraint result into a decoder in the first density estimation model to output the density distribution results of different types of image samples.

[0080] The function of the encoder in the above step is to transform an indefinite-length input sequence into a fixed-length context variable, and encode the input sequence information in the context variable; wherein the encoder can be a recurrent neural network.

[0081] The decoder in the above step is used to decode the input sequence information of the encoder to output a target sequence; wherein the decoder can be a recurrent neural network.

[0082] In an optional embodiment, the sample features of different types of image samples can be extracted by a feature extraction network, wherein the sample features can be color features of the image, texture features of the image, shape features of the image, and spatial relationship features of the image.

[0083] In another optional embodiment, the sample feature set can be input into the encoder of the first density estimation model to output encoded feature data, the feature data can be classified by the classifier, and the feature data can be classified and labeled according to the classification result of the feature data; for example, the image definition feature in the feature data can be divided into three categories: high image definition category, normal image definition category, and low image definition category.

[0084] In yet another optional embodiment, the sample feature set can be input into an encoder of the first density estimation model, and encoded feature data is output. The feature data can be clustered, and the feature data can be classified and labeled according to the clustering result of the feature data. For example, the feature data can be clustered into three categories, i.e., a category of high image definition, a category of normal image definition, and a category of low image definition.

[0085] In the above embodiments of the present application, the encoded feature data is subjected to constraint processing by using a classifier. The constraint processing includes: using the classifier to classify the encoded feature data according to a plurality of preset feature classification weights, to obtain different types of feature data sets; and labeling the different types of feature data sets respectively, to obtain constraint results, wherein the constraint results include the feature data after classification and labeling.

[0086] In an optional embodiment, the classifier can be used to classify each feature in the image sample according to the weight of each feature in different categories, to obtain different types of feature data sets, wherein the classification weight of each feature in the classifier can be determined according to the definition.

[0087] Further, the image sample with high definition can be labeled with a high definition label, so that subsequent training of the first density estimation model using the image sample can achieve the effect of classification training, thereby improving the accuracy of the first density estimation model.

[0088] In the above embodiments of the present application, after outputting the density distribution results of different types of image samples, the method further includes at least one of the following: visualizing the density distribution results of different types of image samples; and mapping the density distribution results of different types of image samples to the sample feature set, to output visualized samples.

[0089] In an optional embodiment, the density distribution results can be displayed in the form of data, in the form of a column chart, in the form of a pie chart, in the form of a list, or in the form of a document.

[0090] In another optional embodiment, the density distribution results of different types of image samples are mapped to the sample feature set, and the visualized samples are output, so that the user can view the density distribution results of the image sample when viewing the image sample.

[0091] In yet another optional embodiment, the visualized processed density distribution result and the visualized sample can be output to the user device, so that the user can timely understand the training effect of the first density estimation model on the user device, and adjust the training manner of the first density estimation model.

[0092] In the above embodiments of the present application, after obtaining the density distribution results of different types of image samples, the method further comprises: optimizing the training result of the first density estimation model by using a second density estimation model to obtain an updated density distribution result.

[0093] Specifically, the second density estimation model can be a convolutional neural network model, which is used to identify the density of the target object in the image, thereby outputting the density distribution result of different types of image samples. The second density estimation model uses the density distribution result output by the first density estimation result in the process of identifying the image, so that the updated density distribution result obtained by the second density estimation model is more accurate.

[0094] In the above scheme, the training result of the first density estimation model is optimized by the second density estimation model, which can make the density distribution result more accurate. Specifically, the second density estimation model can re-cluster the density distribution result output by the first density estimation model to obtain a clustering result, and re-label according to the clustering result to form a more accurate density distribution result.

[0095] In the above embodiments of the present application, the training result of the first density estimation model is optimized by using the second density estimation model to obtain an updated density distribution result, which comprises: clustering the density distribution results of different types of image samples to output a clustering result; determining a feature classification weight in the second density estimation model based on the clustering result, wherein the feature classification weight represents the classification manner adopted by the encoder of the second density estimation model when classifying feature data; re-classifying and labeling different types of image samples according to the feature classification weight in the second density estimation model; inputting the re-classified and labeled result into the decoder of the second density estimation model to generate an updated density distribution result.

[0096] In an optional embodiment, the density distribution results of different types of image samples can be clustered to obtain density-high density distribution results, density-general density distribution results and density-low density distribution results. According to the clustering results, the weights of feature classification in the second density estimation model are determined. For example, if the density distribution result indicates that the density of the image sample is high, the weight of the high density in the image sample is high; if the density distribution result indicates that the density of the image sample is general, the weight of the general density in the image sample is general; and if the density distribution result indicates that the density of the image sample is low, the weight of the low density in the image sample is low.

[0097] Further, according to the feature classification weights in the second density estimation model, the labels of different types of image samples are re-set. If the weight of the high density in the density distribution result of the image sample is high, the image sample can be set with a high-density label; if the weight of the general density in the density distribution result of the image sample is high, the image sample can be set with a general-density label; and if the weight of the low density in the density distribution result of the image sample is high, the image sample can be set with a low-density label.

[0098] In the above embodiments of the present application, after the updated density distribution result is generated, the method further includes at least one of the following: visualizing the updated density distribution result; and mapping the updated density distribution result to a sample feature set to output a visualized sample, wherein the sample feature set is obtained by extracting sample features of different types of image samples.

[0099] In an optional embodiment, the updated density distribution result can be displayed in the form of data, in the form of a column chart, in the form of a pie chart, in the form of a list or in the form of a document.

[0100] In another optional embodiment, the updated density distribution result is mapped to the sample feature set, and the visualized sample is output to enable the user to view the density distribution result of the image sample when viewing the image sample.

[0101] In yet another optional embodiment, the visualized density distribution result and the visualized sample can be output to a user device, so that the user can understand the training effect of the first density estimation model in time on the user device, and thus adjust the training method of the first density estimation model.

[0102] In the above embodiments of the present application, after obtaining the training samples composed of at least two different types of image samples, the method further comprises: collecting key data of the image samples by using the convolution layer of the first density estimation model, and taking the key data as the input of the pooling layer of the first density estimation model.

[0103] The first density estimation model in the above step can be a convolutional neural network, wherein each convolutional layer in the convolutional neural network is composed of a plurality of convolutional units, and the parameters of each convolutional unit are obtained by a back propagation algorithm. The purpose of convolution operation is to extract different features of the input, and the first convolutional layer can only extract some low-level features such as edges, lines and angles, and more layers of network can iteratively extract more complex features from low-level features.

[0104] The pooling layer in the above step is sandwiched between the consecutive convolutional layers, and is used to compress the amount of data and parameters. For example, if the input is an image, the main role of the pooling layer is to compress the image.

[0105] The key data in the above step can be pixel features, texture features, structural features, definition, size features, etc. of the image.

[0106] The above steps will be described in detail below Figure 4 A preferred embodiment of the present application is described in detail, which can be executed by a mobile terminal or a server. In the embodiments of the present application, the method is executed by the server as an example.

[0107] As shown in the above Figure 4 The density estimation model with high accuracy is trained by a deterministic domain attention module (DeCA) and an intrinsic domain attention module (InCA), wherein the deterministic domain attention module corresponds to the first density estimation model, and the intrinsic domain attention module corresponds to the second density estimation model.

[0108] In the above Figure 5As shown in the deterministic domain attention module, first, different types of data sets are obtained, wherein the different types of data sets can be SHA, QNRF and SHB, and each data set includes different image samples; and 3-dimensional features of each image sample in different data sets are extracted through a global average pooling layer (GAP, Global Average Pooling), and the extracted 3-dimensional features are encoded through an encoder to obtain feature data; then, the image samples in the data set are classified through a classifier to obtain feature data of different categories, i.e., feature data belonging to range 1, feature data belonging to range 2, and the feature data is set with a digital label according to the weight of the feature data of different categories, wherein 0 represents that the crowd density in the feature data is large, 1 represents that the crowd density in the feature data is general, and 2 represents that the crowd density in the feature data is small; finally, the feature data set with the digital label is decoded through a decoder to obtain a sample vector, and finally, a density distribution result is obtained based on the sample vector, and the density distribution result is visualized for the user to observe. Further, the attention vector can be mapped on the sample feature in the form of dot product to obtain a visualized sample feature, wherein the formula of the dot product is, z is the visualized sample feature obtained, x is the 3-dimensional feature extracted by the global average pooling layer, is the sample vector obtained by the decoder.

[0109] It should be noted that the process of encoding the extracted 3-dimensional feature by using the encoder is an example of using the fully connected layer in the encoder to encode the 3-dimensional feature to obtain the feature data.

[0110] As shown in the deterministic domain attention module, first, different types of data sets are obtained, wherein the different types of data sets can be SHA, QNRF and SHB, and each data set includes different image samples; and 3-dimensional features of each image sample in different data sets are extracted through a global average pooling layer (GAP, Global Average Pooling), and the extracted 3-dimensional features are encoded through an encoder to obtain feature data; then, the image samples in the data set are classified through a classifier to obtain feature data of different categories, i.e., feature data belonging to range 1, feature data belonging to range 2, and the feature data is set with a digital label according to the weight of the feature data of different categories, wherein 0 represents that the crowd density in the feature data is large, 1 represents that the crowd density in the feature data is general, and 2 represents that the crowd density in the feature data is small; finally, the feature data set with the digital label is decoded through a decoder to obtain a sample vector, and finally, a density distribution result is obtained based on the sample vector, and the density distribution result is visualized for the user to observe. Further, the attention vector can be mapped on the sample feature in the form of dot product to obtain a visualized sample feature, wherein the formula of the dot product is, Figure 6 As shown in the essential domain attention module, compared with the deterministic domain attention module, the essential domain attention module will first use the density distribution result of the deterministic domain attention module to cluster to obtain a more accurate clustering result, and reassign a label to all image samples according to the clustering result, which can be called a current label (CL label). It should be noted that the essential domain attention module is different from the deterministic domain attention module in that the output of the essential domain module is supervised learning through a soft classifier (Softmax), while in the deterministic domain attention module, it is not necessary.

[0111] It should be noted that the essential domain attention module and the deterministic domain attention module described above can be learned in a progressive manner, i.e., the deterministic domain attention module is used in the first stage, and after the deterministic domain attention module is trained, the essential domain attention module is used to replace the deterministic domain attention module. The overall model can be called DKPNet, as shown in Figure 7As shown, the basic depth framework module can be an HRNet model (High Resolution Net), and the essential domain attention module and the deterministic domain attention module are inserted into the HRNet model, and an optimized density distribution result can be obtained, wherein an ASPP (atrous spatial pyramid pooling) module is used to learn the knowledge of different receptive fields, wherein r represents the size of the receptive field.

[0112] Further, the learning of the density estimation model can be performed by optimizing the following loss function:

[0113]

[0114] wherein N represents the number of data samples, Y represents the true value of the crowd density map, Y with a superscript represents the predicted value of the density estimation model, lamda represents a hyperparameter, L deca and L MSC represent classification functions (softmax) and mixed classification functions, respectively.

[0115] Embodiment 2

[0116] According to the embodiments of the present application, a data processing method is also provided. It should be noted that the steps shown in the flowchart can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0117] Figure 8 is a flowchart of the data processing method according to Embodiment 2 of the present application. As shown in the flowchart, the method comprises the following steps: Figure 8

[0118] Step S802, obtaining a video to be recognized.

[0119] wherein the video comprises a plurality of images.

[0120] The video to be recognized in the above steps can be a live video, a conference video, a recorded video, etc.

[0121] Step S804, analyzing the images in the video using a first density estimation model to recognize the density of the target object displayed in each image.

[0122] In an alternative embodiment, if the video to be recognized is a live video, the live images in the video can be analyzed by the first density estimation model to recognize the density of the personnel in each live image, so that the manager can manage the live site according to the density of the personnel.

[0123] ​The first density estimation model is trained by using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0124] The first density estimation model in the above step can be the first density estimation model in Embodiment 1, and the acquisition method of the first density estimation model has been described in Embodiment 1, which will not be repeated here.

[0125] In the above embodiments of the present application, before the first density estimation model is used to analyze the to-be-detected image, the method further includes: obtaining training samples composed of at least two different types of image samples, wherein the image density of the image sample represents the display density of the target object in the image sample; and training the first density estimation model by using different types of image samples and obtaining the density distribution results of different types of image samples.

[0126] In the above embodiments of the present application, the training samples composed of at least two different types of image samples are obtained: obtaining an image sample set; obtaining a plurality of image dimension parameters and combining the plurality of image dimension parameters to generate a classification condition, wherein the image dimension parameters include at least one of the following: image background, image scene, density value of the target object displayed in the image, image viewing angle, image shooting distance, occlusion degree and image definition; and classifying the image sample set based on the classification condition to obtain at least two different types of image samples.

[0127] It should be noted that the preferred embodiments involved in the above embodiments of the present application are the same as the schemes, application scenarios and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0128] Embodiment 3

[0129] According to the embodiments of the present application, a data processing method is also provided. It should be noted that the steps shown in the flowchart can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from that shown here.

[0130] Figure 9 is a flowchart of the data processing method according to Embodiment 3 of the present application. As Figure 9 shown, the method includes the following steps:

[0131] Step S902, obtaining a video in a predetermined space within a predetermined time period.

[0132] The video contains products located in the predetermined space within the predetermined time period, and the video includes a plurality of images.

[0133] The predetermined space in the above step can be a space that needs to be live broadcasted. By obtaining the video of the predetermined control in the predetermined time period, the personnel density in the space can be live broadcasted by the management personnel as needed.

[0134] In an optional embodiment, the predetermined space can be a football viewing stand, and the predetermined time period can be the time period of a football match. By obtaining the video of the football viewing stand in the time period of the football match and analyzing the images in the video by using the first density estimation model, the personnel density of the football viewing stand can be live broadcasted by the management personnel, so that the safety management can be performed in time when the personnel density in the viewing stand is too large.

[0135] In step S904, the images in the video are analyzed by using the first density estimation model to identify the display density of the products in each frame of image.

[0136] In the above step, the first density estimation model is trained by using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0137] In step S906, prompt information is output based on the display density of the products in the images, and the prompt information includes at least one of the following: product occlusion, unreasonable product display quantity, and product type error.

[0138] In an optional embodiment, the display density of the products is too large, and at this time, the product occlusion occurs. Therefore, the prompt information of the product occlusion can be output. The display density of the products is too small, and at this time, the unreasonable product display quantity occurs. Therefore, the prompt information of the unreasonable product display quantity can be output. The display density of the products is normal, and at this time, whether the product type is normal can be detected. If the product type is incorrect, the prompt information of the product type error can be output.

[0139] In the above step, the first density estimation model is trained by using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0140] The first density estimation model in the above step can be the first density estimation model in Embodiment 1, and the obtaining method of the first density estimation model has been described in Embodiment 1, which will not be described here.

[0141] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same application scenarios and implementation processes as the schemes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0142] Embodiment 4

[0143] According to the embodiments of the present application, a data processing method is also provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0144] Figure 10 is a flowchart of a data processing method according to Embodiment 4 of the present application. As shown in Figure 10 , the method comprises the following steps:

[0145] Step S1002, displaying a live video to be identified in a live interface.

[0146] Among them, different types of products are played in the live video. The live interface in the above steps can be a live interface in an e-commerce platform, and the live video to be identified can be a live video of a product for sale.

[0147] Step S1004, in response to a detection instruction sensed in the live interface, calling a first density estimation model.

[0148] The detection instruction in the above steps can be obtained by the user pressing the keys on the screen of the electronic device.

[0149] In an alternative embodiment, in response to the detection instruction sensed in the live interface, it is indicated that the anchor needs to confirm whether the product placement density in the current live interface is normal, and by calling the first density estimation model, the density of different types of products in the live video can be identified, so as to remind the anchor whether to need to re-place the products.

[0150] In another alternative embodiment, the detection instruction can also be a voice instruction, for example, detecting whether the live picture is normal.

[0151] Step S1006, displaying the density of products in each frame of image in the live video in the live interface.

[0152] Among them, the first density estimation model is used to analyze the images in the video, and the density of the target object displayed in each frame of image is identified.

[0153] Among them, different types of image samples are used to train the first density estimation model, the image density displayed in different types of image samples is in different density interval values, and in the training process, different types of image samples are classified and labeled.

[0154] The first density estimation model in the above step can be the first density estimation model in Embodiment 1, and the method for obtaining the first density estimation model has been described in Embodiment 1, which will not be repeated here.

[0155] In an optional embodiment, by displaying the density of the product in each frame of image in the live video in the live interface, the user can confirm whether the product needs to be adjusted in the arrangement manner to achieve the expected live effect.

[0156] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same scheme, application scenario and implementation process as provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0157] Embodiment 5

[0158] According to the embodiments of the present application, a data processing method is also provided. It should be noted that the steps shown in the flowchart can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from here.

[0159] Figure 11 is a flowchart of the data processing method according to Embodiment 5 of the present application. As shown in Figure 11 , the method comprises the following steps:

[0160] Step S1102, the cloud server receives an identification message from the client.

[0161] The identification message can carry identification information representing the video to be identified. The identification message in the above step can be an identification instruction from the client, and the identification information it carries can be the name of the video, or the storage address of the video.

[0162] Step S1104, the cloud server obtains the video to be identified based on the identification information.

[0163] In an optional embodiment, the video to be identified can be obtained according to the name of the video or the storage address of the video.

[0164] Step S1106, the cloud server analyzes the images in the video using a first density estimation model to identify the density of the target object displayed in each frame of image.

[0165] The first density estimation model in the above step can be the first density estimation model in Embodiment 1, and the method for obtaining the first density estimation model has been described in Embodiment 1, which will not be repeated here.

[0166] Step S1108, the cloud server returns the recognition result to the client.

[0167] The first density estimation model is trained by using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0168] The recognition result in the above steps can be the density condition of the target object displayed in each frame of image, for example, the density of the target object is too large, the density of the target object is normal, the density of the target object is too small, etc.

[0169] In an optional embodiment, the cloud server can return the recognition result to the client in the form of a document, audio, or video for the client to view.

[0170] In the above embodiments of the present application, before the cloud server analyzes the to-be-detected image by using the first density estimation model, the method further includes: the cloud server acquires training samples composed of at least two types of image samples, wherein the image density of the image sample represents the display density of the target object in the image sample; the cloud server trains the first density estimation model by using different types of image samples and acquires the density distribution result of different types of image samples.

[0171] It should be noted that the preferred implementation schemes and embodiments involved in the above embodiments of the present application are the same as the schemes, application scenarios, and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0172] Embodiment 6

[0173] According to the embodiments of the present application, a data processing method is also provided. It should be noted that the steps shown in the flowchart can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0174] Figure 12 is a flowchart of the data processing method according to Embodiment 6 of the present application. As shown in Figure 12 the method includes the following steps:

[0175] Step S1202, displaying the captured vehicle video in the road traffic video interface.

[0176] Each frame of image in the vehicle video contains different types of vehicles.

[0177] The road traffic video interface in the above steps can be a live streaming interface, which displays the vehicle videos to be live streamed. The video can capture vehicle information of vehicles passing through a predetermined road segment within a certain time period, and the video can display different types of vehicles.

[0178] Step S1204: Analyze any frame of the vehicle video using the first density estimation model to identify the density of the vehicles displayed in each frame.

[0179] Step S1206: Based on the density of vehicles displayed in the image, control the road traffic lights to output corresponding control information.

[0180] The control information in the above steps can be control signals for red, yellow, and green lights.

[0181] In one optional embodiment, if the density of vehicles displayed in the image is detected to be too low, the traffic lights can be controlled to output a green light signal; if the density of vehicles displayed in the image is detected to be moderate, the traffic lights can be controlled to output a yellow light signal; if the density of vehicles displayed in the image is detected to be too high, the traffic lights can be controlled to output a red light signal.

[0182] The first density estimation model is trained using different types of image samples. The image density displayed in the different types of image samples is in different density ranges. During the training process, the different types of image samples are classified and labeled.

[0183] The first density estimation model in the above steps can be the first density estimation model in Example 1. The method for obtaining the first density estimation model has been described in Example 1 and will not be repeated here.

[0184] In one alternative embodiment, a first density estimation model can be used to analyze images in vehicle videos and identify the density of vehicles displayed in each frame. By displaying the vehicle density on an interface viewed by management personnel, management personnel can easily manage traffic based on the vehicle density.

[0185] It should be noted that the preferred implementation schemes involved in the above embodiments of this application are the same as the schemes, application scenarios and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0186] Example 7

[0187] According to embodiments of this application, a data processing apparatus for implementing the above-described data processing method is also provided, such as... Figure 13 As shown, the device 1300 includes: an acquisition module 1302 and a training module 1304.

[0188] The obtaining module 1302 is configured to obtain training samples composed of at least two different types of image samples, wherein the image density displayed in the different types of image samples is at different density interval values, and the image density of the image sample represents the display density of the target object in the image sample. The training module 1304 is configured to train the first density estimation model using the different types of image samples and obtain the density distribution results of the different types of image samples, wherein the different types of image samples are classified and labeled during the training process.

[0189] It should be noted that the obtaining module 1302 and the training module 1304 correspond to steps S202 to S204 in Embodiment 1, and the two modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules can run in the computer terminal 10 provided in Embodiment 1 as part of the device.

[0190] In the above embodiments of the present application, the obtaining module includes a first obtaining unit, a second obtaining unit, and a classification unit.

[0191] The first obtaining unit is configured to obtain an image sample set. The second obtaining unit is configured to obtain a plurality of image dimension parameters and combine the plurality of image dimension parameters to generate a classification condition, wherein the image dimension parameters include at least one of the following: image background, image scene, density value of a target object displayed in the image, image viewing angle, image shooting distance, occlusion degree, and image definition. The classification unit is configured to classify the image sample set based on the classification condition to obtain at least two different types of image samples.

[0192] In the above embodiments of the present application, the training module includes an extraction unit, a first output unit, a processing unit, and a second output unit.

[0193] The extraction unit is configured to extract sample features of the different types of image samples to obtain a sample feature set. The first output unit is configured to input the sample feature set into an encoder in the first density estimation model and output encoded feature data. The processing unit is configured to constrain the encoded feature data using a classifier, wherein the constraint processing represents classifying and labeling the encoded feature data. The second output unit is configured to input the constraint result into a decoder in the first density estimation model and output the density distribution results of the different types of image samples.

[0194] In the above embodiments of the present application, the processing unit includes a classification subunit and a labeling subunit.

[0195] The classification subunit is configured to classify the coded feature data by using a classifier according to preset feature classification weights, to obtain feature data sets of different types; and the marking subunit is configured to mark the feature data sets of different types respectively, to obtain constraint results, wherein the constraint results include feature data after classification marking.

[0196] In the above embodiments of the present application, the device further includes a first processing module and a mapping module.

[0197] The second processing module is configured to perform visual processing on the density distribution results of the image samples of different types; and the mapping module is configured to map the density distribution results of the image samples of different types to a sample feature set, to output visual samples.

[0198] In the above embodiments of the present application, the device further includes a second processing module.

[0199] The second processing module is configured to optimize the training results of the first density estimation model by using a second density estimation model, to obtain updated density distribution results.

[0200] In the above embodiments of the present application, the second processing module includes an output unit, a determination unit, a marking unit and a generation unit.

[0201] The output unit is configured to cluster the density distribution results of the image samples of different types, to output clustering results; the determination unit is configured to determine feature classification weights in the second density estimation model based on the clustering results, wherein the feature classification weights represent classification manners adopted by an encoder of the second density estimation model when classifying feature data; the marking unit is configured to reclassify and mark the image samples of different types according to the feature classification weights in the second density estimation model; and the generation unit is configured to input the reclassified and marked results to a decoder in the second density estimation model, to generate updated density distribution results.

[0202] In the above embodiments of the present application, the device further includes a third processing module and an output module.

[0203] The third processing module is configured to perform visual processing on the updated density distribution results; and the output module is configured to map the updated density distribution results to a sample feature set, to output visual samples, wherein the sample feature set is obtained by extracting sample features of the image samples of different types.

[0204] In the above embodiments of the present application, the device further includes a collection module.

[0205] The collection module is configured to collect key data of the image sample by using a convolutional layer of the first density estimation model, and input the key data into a pooling layer of the first density estimation model.

[0206] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same application scenarios and implementation processes as the schemes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0207] Embodiment 8

[0208] According to the embodiments of the present application, a data processing device for implementing the data processing method is also provided, as shown in the figure, the device 1400 includes an acquisition module 1402, an identification module 1404, and an output module 1406. Figure 14

[0209] The first acquisition module 1402 is configured to acquire a video to be identified, wherein the video includes multiple images; the identification module 1404 is configured to analyze the images in the video by using a first density estimation model to identify the density of a target object displayed in each image; and the output module 1406 is configured to output prompt information based on the display density of a product in the image, wherein the prompt information includes at least one of the following: product occlusion, unreasonable product display quantity, and product type error; wherein the first density estimation model is trained by using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0210] It should be noted that the first acquisition module 1402, the identification module 1404, and the output module 1406 correspond to steps S802 to S806 in Embodiment 2, and the two modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules as part of the device can run in the computer terminal 10 provided in Embodiment 1.

[0211] In the above embodiments of the present application, the device further includes a second acquisition module and a training module.

[0212] The second acquisition module is configured to acquire training samples composed of at least two different types of image samples, wherein the image density of the image sample represents the display density of the target object in the image sample; and the training module is configured to train the first density estimation model by using different types of image samples, and obtain the density distribution results of the different types of image samples.

[0213] In the above embodiments of the present application, the second acquisition module includes a first acquisition unit, a second acquisition unit, and a classification unit. ​

[0214] The first obtaining unit is configured to obtain an image sample set; the second obtaining unit is configured to obtain a plurality of image dimension parameters, and combine the plurality of image dimension parameters to generate a classification condition, wherein the image dimension parameters include at least one of the following: image background, image scene, density value of a target object displayed in the image, image perspective, image shooting distance, occlusion degree, and image definition; and the classification unit is configured to classify the image sample set based on the classification condition to obtain at least two different types of image samples.

[0215] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same scheme, application scenario and implementation process as provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0216] Embodiment 9

[0217] According to the embodiments of the present application, a data processing device for implementing the above-mentioned data processing method is also provided, as shown in Figure 15 The device 1500 includes an obtaining module 1502, an identifying module 1504, and an output module 1506.

[0218] The obtaining module 1502 is configured to obtain a video in a predetermined space within a predetermined time period, wherein the video contains products located in the predetermined space within the predetermined time period, and the video includes a plurality of images; the identifying module 1504 is configured to analyze the images in the video using a first density estimation model to identify the display density of the products in each image; and the output module 1506 is configured to output prompt information based on the display density of the products in the images, wherein the prompt information includes at least one of the following: product occlusion, unreasonable product display quantity, and product type error; wherein the first density estimation model is trained using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled during the training process.

[0219] It should be noted that the obtaining module 1502, the identifying module 1504, and the output module 1506 correspond to steps S902 to S906 in Embodiment 3, and the two modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0220] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same scheme, application scenario and implementation process as provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0221] Embodiment 10

[0222] According to the embodiments of the present application, a data processing device for implementing the data processing method is also provided, as shown in Figure 16 The device 1600 includes a display module 1602, a response module 1604, and a display module 1606.

[0223] The display module 1602 is configured to display a live video to be identified in a live interface, wherein different types of products are played in the live video. The response module 1604 is configured to respond to a detection instruction sensed in the live interface and call a first density estimation model. The display module 1606 is configured to display the density of products in each frame of image in the live video in the live interface. The first density estimation model is used to analyze the images in the video and identify the density of the target object displayed in each frame of image. Different types of image samples are used to train the first density estimation model. The image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0224] It should be noted that the display module 1602, the response module 1604, and the display module 1606 correspond to steps S1002 to S1006 in Embodiment 4. The two modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules as part of the device can run in the computer terminal 10 provided in Embodiment 1.

[0225] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same scheme, application scenario, and implementation process as the scheme provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0226] Embodiment 11

[0227] According to the embodiments of the present application, a data processing device for implementing the data processing method is also provided, as shown in Figure 17 The device 1700 includes a receiving module 1702, a first obtaining module 1704, an analysis module 1706, and a returning module 1708.

[0228] The receiving module 1702 is configured to receive an identification message from a client, where the identification message carries identification information representing a video to be identified; the first obtaining module 1704 is configured to obtain the video to be identified based on the identification information; the analysis module 1706 is configured to analyze images in the video by using a first density estimation model to identify the density of a target object displayed in each frame of image; and the returning module 1708 is configured to return an identification result to the client. The first density estimation model is trained by using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0229] It should be noted that the receiving module 1702, the first obtaining module 1704, the analysis module 1706, and the returning module 1708 correspond to steps S1102 to S1108 in Embodiment 5, and the two modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules can run in the computer terminal 10 provided in Embodiment 1 as part of the device.

[0230] In the above embodiments of the present application, the device further includes a second obtaining module and a training module.

[0231] The second obtaining module is configured to obtain training samples composed of at least two different types of image samples, where the image density of the image samples represents the display density of the target object in the image samples; and the training module is configured to train the first density estimation model by using the different types of image samples and obtain the density distribution results of the different types of image samples.

[0232] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same scheme, application scenario, and implementation process as those provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0233] Embodiment 12

[0234] According to the embodiments of the present application, a data processing device for implementing the data processing method is also provided, as shown in Figure 18 The device 1800 includes a display module 1802, an identification module 1804, and a control module 1806.

[0235] The display module 1802 is configured to display a vehicle video to be live broadcast in a road traffic video interface, wherein different types of vehicles are displayed in the vehicle video. The identification module 1804 is configured to analyze images in the vehicle video by using a first density estimation model to identify the density of the vehicles displayed in each frame of image. The control module 1806 is configured to control a road traffic light to output corresponding control information based on the density of the vehicles displayed in the image. The first density estimation model is trained by using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0236] It should be noted that the display module 1802 and the identification module 1804 correspond to steps S1102 to S1104 in Embodiment 7, and the two modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules can run in the computer terminal 10 provided in Embodiment 1 as part of the device.

[0237] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same scheme, application scenario, and implementation process as the scheme provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0238] Embodiment 13

[0239] According to the embodiments of the present application, a data processing method is also provided. It should be noted that the steps shown in the flowchart can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in a different order.

[0240] Figure 19 is a flowchart of the data processing method according to Embodiment 13 of the present application. As shown in Figure 19 , the method comprises the following steps:

[0241] Step S1902, displaying a video to be identified in an interactive interface.

[0242] The interactive interface provides at least one control for interaction, and the video includes multiple frames of images.

[0243] The interactive interface in the above steps can be the interface of an interactive tablet, a mobile phone, or a computer, wherein a user can operate on the interactive interface.

[0244] The control in the above steps can be a floating control or a fixed control on the interactive interface.

[0245] Step S1904, the trigger control generates an instruction to call the first density estimation model by responding to the instruction.

[0246] In an optional embodiment, the above-mentioned control can be a density estimation function control. When a user watches a video, the user can trigger the control to call the first density estimation model, so that the first density estimation model can estimate the density of a target object in a current video frame.

[0247] In another optional embodiment, the above-mentioned control can be a voice control function control. When a user watches a video, the user can trigger the control to start the voice control function of the device and speak. When the voice of the user is a preset voice, a voice instruction is generated according to the voice of the user, and the first density estimation model is called by responding to the voice instruction. The preset voice can be please identify, please identify density, identify density, and the like.

[0248] Step S1906, the first density estimation model is used to analyze images in a video, and the density of a target object displayed in each image is identified.

[0249] Step S1908, the density of the target object displayed in each image is displayed in a display interface.

[0250] In an optional embodiment, the density of the target object displayed in each image can be displayed in the display interface in the form of text or numbers.

[0251] The first density estimation model is trained by using different types of image samples. The image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0252] It should be noted that the preferred implementation schemes and embodiments involved in the above embodiments of the present application are the same as the schemes and application scenarios and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0253] Embodiment 14

[0254] According to the embodiments of the present application, a data processing apparatus for implementing the above-mentioned data processing method is also provided, as shown in Figure 20 The apparatus 2000 includes a display module 2002, a trigger module 2004, an identification module 2006, and a display module 2008.

[0255] The display module 2002 is configured to display a video to be recognized in an interactive interface, wherein the interactive interface provides at least one control for interaction, and the video includes multiple images; the trigger module 2004 is configured to trigger the control to generate an instruction, and to call a first density estimation model in response to the instruction; the recognition module 2006 is configured to analyze the images in the video by using the first density estimation model, and to recognize the density of a target object displayed in each image; and the display module 2008 is configured to display the density of the target object displayed in each image in a display interface. The first density estimation model is trained by using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0256] It should be noted that the display module 2002, the trigger module 2004, the recognition module 2006, and the display module 2008 correspond to steps S1902 to S1908 in Embodiment 13, and the four modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules can run in the computer terminal 10 provided in Embodiment 1 as part of the device.

[0257] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same scheme, application scenario, and implementation process as those provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0258] Embodiment 15

[0259] According to the embodiments of the present application, a data processing system is also provided, which includes:

[0260] a processor; and

[0261] a memory connected with the processor, configured to provide the processor with instructions for processing the following processing steps: obtaining training samples composed of at least two different types of image samples, wherein the image density displayed in the different types of image samples is in different density interval values, and the image density of the image sample represents the display density of a target object in the image sample; training a first density estimation model by using the different types of image samples, and obtaining the density distribution results of the different types of image samples, wherein the different types of image samples are classified and labeled in the training process.

[0262] It should be noted that the preferred embodiments involved in the above embodiments of the present application have the same scheme, application scenario, and implementation process as those provided in Embodiment 1, but are not limited to the scheme provided in Embodiment 1.

[0263] Embodiment 16

[0264] Embodiments of this application may provide a computer terminal, which may be any computer terminal device in a group of computer terminals. Optionally, in this embodiment, the aforementioned computer terminal may also be replaced by a mobile terminal or other terminal device.

[0265] Optionally, in this embodiment, the computer terminal may be located in at least one of a plurality of network devices in a computer network.

[0266] In this embodiment, the computer terminal described above can execute the program code for the following steps in the image processing method:

[0267] Optionally, Figure 19 This is a structural block diagram of a computer terminal according to an embodiment of this application. Figure 19 As shown, the computer terminal A may include one or more (only one is shown in the figure) processors 1902 and memory 1904.

[0268] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the data processing methods and apparatus in this embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned image processing method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to terminal A via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0269] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: acquiring training samples consisting of at least two different types of image samples, wherein the image density displayed in the different types of image samples is in different density ranges, and the image density of the image samples represents the display density of the target object in the image samples; training a first density estimation model using the training samples and obtaining the density distribution results of the different types of image samples, wherein, during the training process, the different types of image samples are classified and labeled.

[0270] Optionally, the processor can further execute program codes of the following steps: obtaining a set of image samples; obtaining a plurality of image dimension parameters and combining the plurality of image dimension parameters to generate a classification condition, wherein the image dimension parameters comprise at least one of the following: image background, image scene, density value of a target object displayed in the image, image perspective, image shooting distance, occlusion degree, and image definition; and classifying the set of image samples based on the classification condition to obtain at least two different types of image samples.

[0271] Optionally, the processor can further execute program codes of the following steps: extracting sample features of the different types of image samples to obtain a set of sample features; inputting the set of sample features into an encoder in the first density estimation model to output encoded feature data; performing constraint processing on the encoded feature data using a classifier, wherein the constraint processing represents classification labeling of the encoded feature data; and inputting the constraint result into a decoder in the first density estimation model to output density distribution results of the different types of image samples.

[0272] Optionally, the processor can further execute program codes of the following steps: performing classification on the encoded feature data using the classifier according to a plurality of preset feature classification weights to obtain a plurality of different types of feature data sets; and labeling the plurality of different types of feature data sets respectively to obtain constraint results, wherein the constraint results comprise the feature data after classification labeling.

[0273] Optionally, the processor can further execute program codes of the following steps: performing visualization processing on the density distribution results of the different types of image samples; and mapping the density distribution results of the different types of image samples onto the set of sample features to output visualized samples.

[0274] Optionally, the processor can further execute program codes of the following steps: performing optimization processing on the training results of the first density estimation model using a second density estimation model to obtain updated density distribution results.

[0275] Optionally, the processor can further execute program codes of the following steps: clustering the density distribution results of the different types of image samples to output clustering results; determining feature classification weights in the second density estimation model based on the clustering results, wherein the feature classification weights represent a classification manner used by an encoder in the second density estimation model when classifying feature data; re-labeling the different types of image samples according to the feature classification weights in the second density estimation model; inputting the re-labeled results into a decoder in the second density estimation model to generate updated density distribution results.

[0276] Optionally, the processor can further execute program codes of the following steps: visualizing the updated density distribution result; and mapping the updated density distribution result to the sample feature set to output a visualized sample, wherein the sample feature set is obtained by extracting sample features of different types of image samples.

[0277] Optionally, the processor can further execute program codes of the following steps: collecting key data of the image sample by using the convolution layer of the first density estimation model, and taking the key data as an input of the pooling layer of the first density estimation model.

[0278] The processor can call information and application programs stored in the memory through the transmission device to execute the following steps: obtaining a video to be recognized, wherein the video comprises multiple image frames; and analyzing the image in the video by using the first density estimation model to recognize the density of the target object displayed in each image frame; wherein the first density estimation model is trained by using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled during the training process.

[0279] Optionally, the processor can further execute program codes of the following steps: obtaining training samples composed of at least two different types of image samples, wherein the image density of the image sample represents the display density of the target object in the image sample; training the first density estimation model by using the different types of image samples, and obtaining density distribution results of the different types of image samples.

[0280] Optionally, the processor can further execute program codes of the following steps: obtaining an image sample set; obtaining multiple image dimension parameters and combining the multiple image dimension parameters to generate a classification condition, wherein the image dimension parameters comprise at least one of the following: image background, image scene, density value of the target object displayed in the image, image perspective, image shooting distance, occlusion degree, and image definition; and classifying the image sample set based on the classification condition to obtain at least two different types of image samples.

[0281] The processor can call information and application programs stored in the memory through the transmission device to execute the following steps: acquiring a video in a predetermined space within a predetermined time period, wherein the video contains products located in the predetermined space within the predetermined time period, and the video includes multiple frames of images; analyzing the images in the video using a first density estimation model to identify the display density of the products in each frame of image; and outputting prompt information based on the display density of the products in the image, the prompt information including at least one of the following: product occlusion, unreasonable product display quantity, and product type error; wherein the first density estimation model is trained using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled during the training process.

[0282] The processor can call information and application programs stored in the memory through the transmission device to execute the following steps: displaying a live video to be identified in a live interface, wherein different types of products are played in the live video; in response to a detection instruction sensed in the live interface, calling a first density estimation model; and displaying the density of the products in each frame of image in the live video in the live interface, wherein the first density estimation model is used to analyze the images in the video to identify the density of the target object displayed in each frame of image; wherein the first density estimation model is trained using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled during the training process.

[0283] The processor can call information and application programs stored in the memory through the transmission device to execute the following steps: a cloud server receives an identification message from a client, wherein the identification message carries identification information representing a video to be identified; the cloud server obtains the video to be identified based on the identification information; the cloud server analyzes the images in the video using a first density estimation model to identify the density of the target object displayed in each frame of image; and the cloud server returns the identification result to the client; wherein the first density estimation model is trained using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled during the training process.

[0284] Optionally, the processor can further execute the program code of the following steps: the cloud server obtains training samples composed of at least two different types of image samples, wherein the image density of the image samples represents the display density of the target object in the image samples; the cloud server trains the first density estimation model using different types of image samples and obtains the density distribution results of the different types of image samples.

[0285] The processor can call information and application programs stored in the memory through the transmission device to execute the following steps: displaying a captured vehicle video in a road traffic video interface, wherein each frame of image in the vehicle video displays different types of vehicles; using a first density estimation model to analyze any frame of image in the vehicle video to identify the density of the vehicles displayed in each frame of image; and controlling the road traffic light to output corresponding control information based on the density of the vehicles displayed in the image; wherein the first density estimation model is trained using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled during the training process.

[0286] The processor can call information and application programs stored in the memory through the transmission device to execute the following steps: displaying a captured vehicle video in a road traffic video interface, wherein each frame of image in the vehicle video displays different types of vehicles; using a first density estimation model to analyze any frame of image in the vehicle video to identify the density of the vehicles displayed in each frame of image; and controlling the road traffic light to output corresponding control information based on the density of the vehicles displayed in the image; wherein the first density estimation model is trained using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled during the training process.

[0287] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the present application is not limited to the order of the actions described, because according to the present application, certain steps can be performed in other order or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0288] Those skilled in the art can understand that Figure 21 The structure shown is only schematic, and the computer terminal can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, and the like. Figure 21 It does not limit the structure of the above-mentioned electronic device. For example, the computer terminal A can further include more or fewer components (such as a network interface, a display device, etc.) than Figure 21 shown, or have a different configuration from Figure 21 shown.

[0289] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiments can be completed by instructing the terminal device related hardware through a program, and the program can be stored in a computer readable storage medium, which can include a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0290] Embodiment 17

[0291] The embodiments of the present application can provide a computer terminal, which can be any one of the computer terminal devices in the computer terminal group. Alternatively, in the embodiments, the computer terminal can also be replaced by a mobile terminal or other terminal device.

[0292] Alternatively, in the embodiments, the storage medium is configured to store program code for performing the following steps: obtaining training samples composed of at least two different types of image samples, wherein the image density displayed in the different types of image samples is in different density interval values, and the image density of the image sample represents the display density of the target object in the image sample; training the first density estimation model using the training samples, and obtaining the density distribution results of the different types of image samples, wherein the different types of image samples are classified and labeled during the training process.

[0293] Alternatively, the storage medium is further configured to store program code for performing the following steps: obtaining an image sample set; obtaining a plurality of image dimension parameters and combining the plurality of image dimension parameters to generate a classification condition, wherein the image dimension parameters include at least one of the following: image background, image scene, density value of the target object displayed in the image, image viewing angle, image shooting distance, occlusion degree and image definition; and classifying the image sample set based on the classification condition to obtain at least two different types of image samples.

[0294] Alternatively, the storage medium is further configured to store program code for performing the following steps: extracting sample features of the different types of image samples to obtain a sample feature set; inputting the sample feature set into an encoder in the first density estimation model to output encoded feature data; using a classifier to perform constraint processing on the encoded feature data, wherein the constraint processing represents classifying and labeling the encoded feature data; and inputting the constraint result into a decoder in the first density estimation model to output the density distribution results of the different types of image samples.

[0295] Optionally, the storage medium is further configured to store program code for performing the following steps: classifying the encoded feature data by using the classifier according to preset feature classification weights, to obtain feature data sets of different types; and marking the feature data sets of different types respectively, to obtain constraint results, wherein the constraint results include the feature data after classification marking.

[0296] Optionally, the storage medium is further configured to store program code for performing the following steps: visualizing the density distribution results of the image samples of different types; and mapping the density distribution results of the image samples of different types to the sample feature set, to output visualized samples.

[0297] Optionally, the storage medium is further configured to store program code for performing the following steps: optimizing the training results of the first density estimation model by using a second density estimation model, to obtain updated density distribution results.

[0298] Optionally, the storage medium is further configured to store program code for performing the following steps: clustering the density distribution results of the image samples of different types, to output clustering results; determining feature classification weights in the second density estimation model based on the clustering results, wherein the feature classification weights represent a classification manner adopted by an encoder of the second density estimation model when classifying feature data; re-marking the image samples of different types according to the feature classification weights in the second density estimation model; and inputting the re-marked results to a decoder in the second density estimation model, to generate updated density distribution results.

[0299] Optionally, the storage medium is further configured to store program code for performing the following steps: visualizing the updated density distribution results; and mapping the updated density distribution results to the sample feature set, to output visualized samples, wherein the sample feature set is obtained by extracting sample features of the image samples of different types.

[0300] Optionally, the storage medium is further configured to store program code for performing the following steps: collecting key data of the image samples by using a convolution layer of the first density estimation model, and taking the key data as an input of a pooling layer of the first density estimation model.

[0301] Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: obtaining a video to be recognized, wherein the video comprises a plurality of images; and using the first density estimation model to analyze the images in the video to identify the density of the target object displayed in each image; wherein the first density estimation model is trained using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0302] Optionally, the storage medium is further configured to store program code for performing the following steps: obtaining training samples composed of at least two different types of image samples, wherein the image density of the image samples represents the display density of the target object in the image samples; training the first density estimation model using the different types of image samples and obtaining the density distribution results of the different types of image samples.

[0303] Optionally, the storage medium is further configured to store program code for performing the following steps: obtaining a set of image samples; obtaining a plurality of image dimension parameters and combining the plurality of image dimension parameters to generate a classification condition, wherein the image dimension parameters comprise at least one of the following: image background, image scene, density value of the target object displayed in the image, image perspective, image shooting distance, occlusion degree, and image definition; and classifying the set of image samples based on the classification condition to obtain at least two different types of image samples.

[0304] Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: obtaining a video in a predetermined space within a predetermined time period, wherein the video contains products located in the predetermined space within the predetermined time period, and the video comprises a plurality of images; using the first density estimation model to analyze the images in the video to identify the display density of the products in each image; and outputting prompt information based on the display density of the products in the images, wherein the prompt information comprises at least one of the following: product occlusion, unreasonable product display quantity, and product type error; wherein the first density estimation model is trained using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0305] Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: displaying a live video to be identified in a live interface, wherein different types of products are played in the live video; in response to a detection instruction sensed in the live interface, calling a first density estimation model; and displaying the density of products in each frame of image in the live video in the live interface, wherein the first density estimation model is used to analyze the image in the video and identify the density of the target object displayed in each frame of image; wherein the first density estimation model is trained by using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0306] Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: receiving an identification message from a client by a cloud server, wherein the identification message carries identification information representing a video to be identified; obtaining the video to be identified by the cloud server based on the identification information; analyzing the image in the video by the cloud server using the first density estimation model, and identifying the density of the target object displayed in each frame of image; and returning the identification result to the client by the cloud server; wherein the first density estimation model is trained by using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0307] Optionally, the storage medium is further configured to store program code for performing the following steps: obtaining training samples composed of at least two different types of image samples by a cloud server, wherein the image density of the image sample represents the display density of the target object in the image sample; training the first density estimation model by the cloud server using different types of image samples, and obtaining the density distribution result of different types of image samples.

[0308] Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: displaying a captured vehicle video in a road traffic video interface, wherein different types of vehicles are displayed in each frame of image included in the vehicle video; analyzing any frame of image in the vehicle video by using the first density estimation model, and identifying the density of the vehicle displayed in each frame of image; and controlling the road traffic light to output corresponding control information based on the density of the vehicle displayed in the image; wherein the first density estimation model is trained by using different types of image samples, the image density displayed in the different types of image samples is in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0309] Optionally, in the embodiment, the storage medium is configured to store program code for performing the following steps: displaying a video to be recognized in an interactive interface, wherein the interactive interface provides at least one control for interaction, and the video includes multiple images; triggering the control to generate an instruction, and invoking a first density estimation model in response to the instruction; analyzing the images in the video by using the first density estimation model to recognize the density of a target object displayed in each image; and displaying the density of the target object displayed in each image in the display interface; wherein the first density estimation model is trained by using different types of image samples, the images displayed in the different types of image samples are in different density interval values, and the different types of image samples are classified and labeled in the training process.

[0310] The above sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0311] In the above embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0312] In the several embodiments provided by the present application, it should be understood that the disclosed technology can be implemented in other ways. Of course, the embodiment described above is only illustrative, and the division of units is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, unit or module, and can be electrical or other forms.

[0313] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiment.

[0314] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0315] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0316] The above is only the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should be considered as the protection scope of the present application.

Claims

1. A method of processing data, characterized by, The method comprises: obtaining training samples composed of at least two different types of image samples, wherein the image density shown in the different types of image samples is at different density interval values; training a first density estimation model using the different types of image samples, and obtaining density distribution results of the different types of image samples, wherein the different types of image samples are classified and labeled during the training process; wherein training the first density estimation model using the different types of image samples and obtaining the density distribution results of the different types of image samples comprises: extracting sample features of the different types of image samples to obtain a sample feature set; encoding the sample feature set to obtain encoded feature data; performing constraint processing on the encoded feature data, wherein the constraint processing represents classification and labeling of the encoded feature data; and decoding the constraint result to obtain the density distribution results of the different types of image samples.

2. The method of claim 1, wherein, Obtaining training samples composed of at least two different types of image samples comprises: obtaining an image sample set; obtaining a plurality of image dimension parameters and combining the plurality of image dimension parameters to generate a classification condition, wherein the image dimension parameters include at least one of the following: image background, image scene, density value of a target object shown in the image, image perspective, image shooting distance, occlusion degree, and image clarity; classifying the image sample set based on the classification condition to obtain the at least two different types of image samples.

3. The method of claim 1, wherein, Encoding the sample feature set to obtain encoded feature data; performing constraint processing on the encoded feature data; decoding the constraint result to obtain the density distribution results of the different types of image samples, comprising: inputting the sample feature set into an encoder in the first density estimation model to output encoded feature data; using a classifier to perform constraint processing on the encoded feature data, wherein the constraint processing represents classification and labeling of the encoded feature data; inputting the constraint result into a decoder in the first density estimation model to output the density distribution results of the different types of image samples.

4. The method of claim 3, wherein, Using a classifier to perform constraint processing on the encoded feature data comprises: using a classifier to classify the encoded feature data according to a plurality of preset feature classification weights to obtain different types of feature data sets; labeling the different types of feature data sets respectively to obtain the constraint result, wherein the constraint result includes feature data after the classification and labeling.

5. The method of claim 3, wherein, After outputting the density distribution results of the different types of image samples, the method further comprises at least one of the following: visualizing the density distribution results of the different types of image samples; mapping the density distribution results of the different types of image samples to the sample feature set to output a visualized sample.

6. The method according to any one of claims 1 to 5, characterized in that, After obtaining the density distribution results of the different types of image samples, the method further comprises: The training result of the first density estimation model is optimized by using a second density estimation model to obtain an updated density distribution result.

7. The method of claim 6, wherein, The training result of the first density estimation model is optimized by using a second density estimation model to obtain an updated density distribution result, including: Clustering the density distribution results of the different types of image samples and outputting clustering results; Based on the clustering results, determining feature classification weights in the second density estimation model, wherein the feature classification weights represent the classification method used by the encoder of the second density estimation model when classifying feature data; According to the feature classification weights in the second density estimation model, the different types of image samples are reclassified and labeled; The reclassified and labeled results are input into the decoder of the second density estimation model to generate an updated density distribution result.

8. The method of claim 7, wherein, After generating the updated density distribution result, the method further includes at least one of the following: Visualizing the updated density distribution result; Mapping the updated density distribution result to a sample feature set to output a visualized sample, wherein the sample feature set is obtained by extracting sample features of the different types of image samples.

9. The method of claim 1, wherein, After obtaining training samples composed of at least two different types of image samples, the method further includes: Using the convolution layer of the first density estimation model to collect key data of the image samples, and using the key data as the input of the pooling layer of the first density estimation model.

10. A data processing method characterized by comprising: Including: Obtaining a video to be recognized, wherein the video includes multiple images; Using a first density estimation model to analyze the images in the video and identify the density of the target object displayed in each image, wherein the first density estimation model is trained using the method of any one of claims 1-8. Wherein, the first density estimation model is trained using different types of image samples, the image density displayed in different types of image samples is in different density interval values, and the different types of image samples are classified and labeled during the training process.

11. The method of claim 10, wherein, Before analyzing the image to be detected using the first density estimation model, the method further includes: Obtaining training samples composed of at least two different types of image samples, wherein the image density of the image samples represents the display density of the target object in the image samples; Training the first density estimation model using the different types of image samples and obtaining the density distribution results of the different types of image samples.

12. The method according to claim 10 or 11, characterized in that, Obtaining training samples composed of at least two different types of image samples: Obtaining an image sample set; Obtaining multiple image dimension parameters and combining the multiple image dimension parameters to generate classification conditions, wherein the image dimension parameters include at least one of the following: image background, image scene, density value of the target object displayed in the image, image viewing angle, image shooting distance, occlusion degree, and image clarity; Classifying the image sample set based on the classification conditions to obtain the at least two different types of image samples.

13. A method of processing data, characterized by, Including: Acquire a video in a predetermined space within a predetermined time period, wherein the video contains a product located in the predetermined space within the predetermined time period, and wherein the video comprises a plurality of images; Analyze the images in the video using a first density estimation model to identify the display density of the product in each image, wherein the first density estimation model is trained using the method of any one of claims 1-8; Output prompt information based on the display density of the product in the images, wherein the prompt information comprises at least one of the following: product occlusion, unreasonable product display quantity, and product type error; The first density estimation model is trained using different types of image samples, the image density displayed in different types of image samples is in different density interval values, and the different types of image samples are classified and labeled during the training process.

14. A method of processing data, characterized by, The method comprises: Display a live video to be identified in a live interface, wherein different types of products are played in the live video; In response to a detection instruction sensed in the live interface, a first density estimation model is invoked, wherein the first density estimation model is trained using the method of any one of claims 1-8; Display the density of the product in each image in the live video in the live interface, wherein the first density estimation model is used to analyze the images in the video to identify the density of the target object displayed in each image; The first density estimation model is trained using different types of image samples, the image density displayed in different types of image samples is in different density interval values, and the different types of image samples are classified and labeled during the training process.

15. A method of processing data, characterized by, The method comprises: A cloud server receives an identification message from a client, wherein the identification message carries identification information representing a video to be identified; The cloud server obtains the video to be identified based on the identification information; The cloud server analyzes the images in the video using a first density estimation model to identify the density of the target object displayed in each image, wherein the first density estimation model is trained using the method of any one of claims 1-8; The cloud server returns the identification result to the client; The first density estimation model is trained using different types of image samples, the image density displayed in different types of image samples is in different density interval values, and the different types of image samples are classified and labeled during the training process.

16. The method of claim 15, wherein, Before the cloud server analyzes the images to be detected using the first density estimation model, the method further comprises: The cloud server obtains training samples composed of at least two different types of image samples, wherein the image density of the image samples represents the display density of the target object in the image samples; The cloud server trains the first density estimation model using the different types of image samples and obtains the density distribution results of the different types of image samples.

17. A method of processing data, characterized by, The method comprises: Display the captured vehicle video in the road traffic video interface, wherein each frame of image contained in the vehicle video displays different types of vehicles; Analyze any frame of image in the vehicle video by using the first density estimation model to identify the density of the vehicle displayed in each frame of image, wherein the first density estimation model is trained by using the method in any one of claims 1 to 8; Control the road traffic light to output corresponding control information based on the density of the vehicle displayed in the image; In the training process, the different types of image samples are classified and labeled.

18. A data processing method characterized by, Comprising: Display the video to be identified in the interactive interface, wherein the interactive interface provides at least one control for interaction, and the video includes multiple frames of images; Trigger the control to generate an instruction, and call the first density estimation model in response to the instruction, wherein the first density estimation model is trained by using the method in any one of claims 1 to 8; Analyze the images in the video by using the first density estimation model to identify the density of the target object displayed in each frame of image; Display the density of the target object displayed in each frame of image in the display interface; In the training process, the different types of image samples are classified and labeled.

19. A data processing apparatus, characterized by comprising: Comprising: An acquisition module configured to acquire training samples composed of at least two different types of image samples, wherein the image density displayed in different types of image samples is in different density interval values, and the image density of the image sample represents the display density of the target object in the image sample; A training module configured to train a first density estimation model by using the different types of image samples and obtain a density distribution result of the different types of image samples, wherein, in the training process, the different types of image samples are classified and labeled; The training module is further configured to extract sample features of the different types of image samples to obtain a sample feature set, encode the sample feature set to obtain encoded feature data, perform constraint processing on the encoded feature data, wherein the constraint processing represents classification and labeling of the encoded feature data, and decode the constraint result to obtain the density distribution result of the different types of image samples.

20. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises a stored program, wherein when the program runs, the computer readable storage medium controls the device where the computer readable storage medium is located to execute the data processing method in any one of claims 1 to 18.

21. A computer terminal, characterized in that Comprising a memory and a processor, the processor is configured to run the program stored in the memory, wherein the program runs to execute the data processing method in any one of claims 1 to 18.

22. A data processing system, characterized by Comprising: A processor; And The memory is connected with the processor and is used to provide the processor with instructions for processing the following processing steps: acquiring a training sample composed of at least two different types of image samples, wherein the image density displayed in the different types of image samples is at different density interval values, and the image density of the image sample represents the display density of a target object in the image sample; training a first density estimation model by using the different types of image samples and acquiring a density distribution result of the different types of image samples, wherein the different types of image samples are classified and labeled during the training process. The training of the first density estimation model by using the different types of image samples and the acquisition of the density distribution result of the different types of image samples include: extracting sample features of the different types of image samples to obtain a sample feature set; encoding the sample feature set to obtain encoded feature data; performing constraint processing on the encoded feature data, wherein the constraint processing represents classification and labeling of the encoded feature data; and performing decoding processing on the constraint result to obtain the density distribution result of the different types of image samples.

Images