Image processing method and apparatus, storage medium, and electronic device

Abstract

The application discloses an image processing method and device, a storage medium and an electronic device. The method comprises the following steps: obtaining a target image of a target object, wherein the target image comprises a local image area in which a target event occurs; inputting the target image into an event occurrence time prediction model; performing image feature extraction on the target image through a first convolution layer in the event occurrence time prediction model, so as to obtain local image features matched with the local image area; inputting the local image features into a second convolution layer of the event occurrence time prediction model, so as to obtain target global features output by the second convolution layer; and determining event occurrence time information of the target event of the target object based on the target global features. The application solves the technical problem that the prediction result of the actual occurrence time of the event is not accurate enough in the related art.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer, in particular to an image processing method and device, a storage medium and an electronic device. BACKGROUND

[0002] In daily life scenarios, it is often necessary to predict the actual occurrence time of an event, such as the aging time of a household appliance, the onset time of a patient, the growth time of a plant, etc. At present, the occurrence time of an event generally needs to rely on expert experience. For example, the aging time of a household appliance needs to be estimated by the staff of the household appliance after checking the household appliance device, the onset time of a patient generally needs to be estimated by a doctor according to the test report of the patient, and the growth time of a plant needs to be estimated by an experienced farmer or researcher. The occurrence time of events in different fields needs to rely on the prediction of experts in that field, which requires a high level of expertise of the expert staff. Moreover, the manual prediction method has the problem of low efficiency.

[0003] Therefore, in the related art, the actual occurrence time of an event is predicted by relying on expert experience. Since the reference factors based on the expert experience are not comprehensive enough, or the expert experience itself has defects, the prediction result is not accurate enough.

[0004] Therefore, in view of the above problems, there is currently no effective solution. SUMMARY

[0005] The embodiments of the present application provide an image processing method and device, a storage medium and an electronic device to at least solve the technical problem of inaccurate prediction result of the actual occurrence time of an event in the related art.

[0006] According to an aspect of an embodiment of the present application, an image processing method is provided, comprising: obtaining a target image of a target object, wherein the target image comprises a local image area where a target event occurs; inputting the target image into an event occurrence time prediction model, performing image feature extraction on the target image through a first convolutional layer in the event occurrence time prediction model, to obtain a local image feature matched with the local image area, wherein the local image feature is used to represent the progress of the target object in the target event; inputting the local image feature into a second convolutional layer of the event occurrence time prediction model, to obtain a target global feature output by the second convolutional layer; and determining event occurrence time information of the target object in the target event based on the target global feature.

[0007] Optionally, the target image is input into the event occurrence time prediction model, image features of the target image are extracted through a first convolutional layer in the event occurrence time prediction model, and local image features matched with the local image region are obtained, including: the local image region is identified in the target image through the first convolutional layer, wherein the local image features include image features of the local image region in multiple dimensions; and the image features of the local image region in the multiple dimensions are determined through the first convolutional layer, wherein the target global feature is used to represent the image features of the local image region in the multiple dimensions, the multiple dimensions are matched with the target event, and the image features in the multiple dimensions are used to represent the progress of the target object in the target event.

[0008] Optionally, the event occurrence time information of the target object in the target event is determined based on the target global feature, including: the target global feature is input into a fully connected layer of the event occurrence time prediction model, and a predicted occurrence duration output by the fully connected layer is obtained, wherein the predicted occurrence duration represents a time length between a time point at which the target object starts to occur the target event and a time point at which the target image is captured, and the event occurrence time information includes the predicted occurrence duration.

[0009] Optionally, the event occurrence time information of the target object in the target event is determined based on the target global feature, including: the target global feature is input into a fully connected layer of the event occurrence time prediction model, and a predicted occurrence duration output by the fully connected layer is obtained, wherein the predicted occurrence duration represents a time length between a time point at which the target object starts to occur the target event and a time point at which the target image is captured; a predicted occurrence time point is determined based on the predicted occurrence duration and a pre-acquired capture time point, wherein the capture time point is the time point at which the target image is captured, the predicted occurrence time point represents the time point at which the target object starts to occur the target event, and the event occurrence time information includes the predicted occurrence time.

[0010] Optionally, the target global feature is input into the fully connected layer of the event occurrence time prediction model, and a predicted occurrence duration output by the fully connected layer is obtained, including: a prediction probability of each time length in a preset time length set is determined through the fully connected layer; and the predicted occurrence duration is determined in the time length set according to the prediction probability of each time length.

[0011] Optionally, the method further comprises: obtaining a sample image set, wherein the sample image set comprises a sample image, known event occurrence time information corresponding to the sample image, the sample image is an image obtained by photographing a sample object, and the known event occurrence time information comprises an actual time point at which the sample object occurs the target event or a time length between the actual time point at which the sample object occurs the target event and a time at which the sample image is photographed; training a sample neural network model using the sample image set until a target loss value of the sample neural network model meets a target convergence condition to obtain the event occurrence time prediction, wherein the target loss value is a loss value determined by event occurrence time information output by the sample neural network model and the known event occurrence time information.

[0012] Optionally, the obtaining the sample image set comprises: obtaining a first sample image, a first actual occurrence time point corresponding to the first sample image, a second sample image, and a target time length, wherein the first sample image and the second sample image are images obtained by photographing a first sample object, the first actual occurrence time point represents an actual time point at which the sample object occurs the target event, and the target time length represents a time length between a time point at which the first sample image is photographed and a time point at which the second sample image is photographed; determining a second actual occurrence time point corresponding to the second sample image according to the first actual occurrence time point and the target time length, wherein the second actual occurrence time point represents an actual time point at which the sample object occurs the target event; and wherein the sample image set comprises the first sample image and the first actual occurrence time point having a corresponding relationship and the second sample image and the second actual occurrence time point having a corresponding relationship.

[0013] Optionally, the obtaining the sample image set comprises: obtaining a third sample image and a third actual occurrence time point corresponding to the third sample image, wherein the third actual occurrence time point represents an actual time point at which a second sample object occurs the target event; performing random flipping and / or rotation processing on the third sample image to obtain a fourth sample image; determining an actual occurrence time point corresponding to the fourth sample image as the third actual occurrence time point; and wherein the sample image set comprises the third sample image and the third actual occurrence time point having a corresponding relationship and the fourth sample image and the third actual occurrence time point having a corresponding relationship.

[0014] According to another aspect of the embodiments of the present application, an image processing apparatus is also provided, comprising: an acquisition module configured to acquire a target image of a target object, wherein the target image comprises a local image region in which a target event occurs; a first input module configured to input the target image into an event occurrence time prediction model, and perform image feature extraction on the target image through a first convolution layer in the event occurrence time prediction model to obtain a local image feature matched with the local image region, wherein the local image feature is used to represent a progress of the target event of the target object; a second input module configured to input the local image feature into a second convolution layer of the event occurrence time prediction model to obtain a target global feature output by the second convolution layer; and a determination module configured to determine event occurrence time information of the target event of the target object based on the target global feature.

[0015] According to still another aspect of the embodiments of the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores a computer program, wherein the computer program is configured to execute the image processing method when running.

[0016] According to still another aspect of the embodiments of the present application, an electronic device is also provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the image processing method through the computer program.

[0017] In the embodiments of the present application, the event occurrence time prediction model is used to predict the event occurrence time information of the target event, and the event occurrence time information of the target event is determined according to the local image feature of the target local image region in the target image through the event occurrence time prediction model, so as to achieve the purpose of predicting the time of the target event of the target object, thereby realizing the technical effect of improving the accuracy of the predicted event occurrence time, and further solving the technical problem that the prediction result of the actual event occurrence time is not accurate in the related art. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings, which are included to provide a further understanding of the present application and constitute a part of this application, illustrate embodiments of the present application and together with the description serve to explain the present application. In the drawings:

[0019] Figure 1 is a schematic diagram of an application environment of an optional image processing method according to an embodiment of the present application;

[0020] Figure 2 is a flowchart of an image processing method according to an embodiment of the present application;

[0021] Figure 3 is a component aging prediction flowchart according to an embodiment of the present application;

[0022] Figure 4 is a patient onset time prediction flowchart according to an embodiment of the present application;

[0023] Figure 5 is a brain CT image according to an embodiment of the present application;

[0024] Figure 6 is a VggNet-19 network structure according to an embodiment of the present application;

[0025] Figure 7 is a ResNet-34 network structure according to an embodiment of the present application;

[0026] Figure 8 is a patient onset time prediction flowchart according to an embodiment of the present application; Figure 1 ;

[0027] Figure 9 is a patient onset time prediction flowchart according to an embodiment of the present application; Figure 2 ;

[0028] Figure 10 is a patient onset time prediction flowchart according to an embodiment of the present application; Figure 1 ;

[0029] Figure 11 is a patient onset time prediction flowchart according to an embodiment of the present application; Figure 2 ;

[0030] Figure 12 is a neural network model obtaining local image dimension feature according to an embodiment of the present application;

[0031] Figure 13 is an event occurrence time prediction model training diagram according to an embodiment of the present application;

[0032] Figure 14 is a structure diagram of an optional image processing device according to an embodiment of the present application;

[0033] Figure 15 is a structure diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely in combination with the accompanying drawings in the embodiments of the present application, so that those skilled in the art can better understand the technical solutions of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should belong to the protection scope of the present application.

[0035] It should be noted that the terms "first", "second" and the like in the description and claims of the present application and the above-mentioned accompanying drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units need not be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0036] According to an aspect of the embodiments of the present application, an image processing method is provided. Optionally, as an optional implementation, the above-mentioned image processing method can be applied to, but is not limited to, a system environment as shown in Figure 1 The above-mentioned system environment includes a terminal device 102, a network 110 and a server 112.

[0037] Optionally, in the embodiment, the terminal device 102 can be a terminal device with a shooting function, and can include but is not limited to at least one of the following: a mobile phone (such as an Android mobile phone, an iOS mobile phone, etc.), a notebook computer, a tablet computer, a palm computer, a MID (Mobile Internet Device), a PAD, a desktop computer, a smart television, a video camera, a B-ultrasound machine, an X-ray machine, etc. The terminal device 102 includes a display 108, a processor 106 and a memory 104. The display 108 can be used to display a shot image, the processor 106 is used to process the image, and the memory 104 is used to store data, including but not limited to image data. The network 110 can include but is not limited to a wired network and a wireless network. The wired network includes a local area network, a metropolitan area network and a wide area network. The wireless network includes Bluetooth, WIFI and other wireless communication networks. The server 112 can be a single server, a server cluster composed of multiple servers, or a cloud server. The server includes a database 114 and a processing engine 115. The database 114 is used to store data, including but not limited to image data, and model architecture and parameters of an event occurrence time prediction model. The processing engine 115 is used to process the image. The above is only an example, and the embodiment is not limited in this regard.

[0038] Optionally, as an optional implementation, as shown in Figure 2 the image processing method includes:

[0039] In step S202, a target image of a target object is obtained, and the target image includes a local image area in which a target event occurs;

[0040] In step S204, the target image is input into an event occurrence time prediction model, image feature extraction is performed on the target image by a first convolutional layer in the event occurrence time prediction model, and a local image feature matched with the local image area is obtained, wherein the local image feature is used to represent a progress of the target object in the target event;

[0041] In step S206, the local image feature is input into a second convolutional layer of the event occurrence time prediction model, and a target global feature output by the second convolutional layer is obtained;

[0042] In step S208, event occurrence time information of the target object in the target event is determined based on the target global feature.

[0043] By the above steps, by using the event occurrence time prediction model to predict the event occurrence time information of the target event, the event occurrence time information of the target event is determined according to the local image features of the target local image region in the target image by the event occurrence time prediction model, the purpose of predicting the time of the target event of the target object is achieved, thereby realizing the technical effect of improving the accuracy of the predicted event occurrence time, and further solving the technical problem that the prediction result of the actual event occurrence time is not accurate in the related art.

[0044] The image processing method can be applied to any scene of predicting the occurrence time of an event, including but not limited to: the aging time of an electrical device, the illness time of a patient, the growth time of a plant, etc. The event occurrence time information includes the time point at which the target event begins to occur, and the time length between the time point at which the target event begins to occur and the time point at which the target image is captured.

[0045] As an optional embodiment, taking the prediction of the aging time of an electrical device as an example, the target object can be an electrical device, and the aging time or the time point at which the electrical device begins to age can be predicted. The aging time refers to the time length between the time point at which the electrical device begins to age and the time point at which the target image is captured. The aging time or the aging time point of a certain component in the electrical device can also be predicted. The target event is the aging of the device. In this embodiment, taking the prediction of the aging time of a component in an electrical device as an example, the electrical device can be captured, and the captured image includes the aged component. Since some components have a special position in the electrical device, it is difficult to capture only the aged component. The captured image can include other parts of the electrical device, and the image features of the aged component need to be extracted. The local image region can be the image of the aged component. The event occurrence time prediction model can be a neural network model such as VggNet or ResNet. The VggNet neural network model can be trained using training data. The training data can be aged components, and the components in the training data can be the same component, such as a mainboard circuit. The aging time of the components can be different, for example, the image of a component that has aged for 1 month, a component that has aged for half a year, a component that has aged for 1 year, etc. The training data can be selected according to actual conditions. The above examples are only for illustrating this embodiment and do not limit it.

[0046] As shown in Figure 3 The component aging prediction process according to an optional embodiment of the present application can include the following steps:

[0047] Step S31, obtain images of components of different aging degrees, and label the aging time of the components corresponding to the images. Take the mainboard circuit as an example, a group of mainboard circuit images of different aging degrees are obtained by photographing the mainboard circuit of different aging degrees, and the mainboard circuit images of different aging degrees can be labeled according to the experience of experts, and the aging time of the components in the images is labeled.

[0048] Step S32, using the images of components of different aging degrees and the corresponding aging time as training data, training the initial neural network model to obtain an event occurrence time prediction model. The event occurrence time prediction model can be a deep neural network model, for example, it can be VggNet, ResNet, etc. The network model is trained using the training data, and the model parameters are repeatedly adjusted during the training process until the loss function between the aging time output by the model and the known aging time of the components in the training data meets the preset convergence condition. The preset convergence condition can use a convergence function, and the output value of the convergence function between the aging time output by the model and the known aging time of the components is in a preset range. Stop training the network model to obtain the event occurrence time prediction model.

[0049] Step S33, using the event occurrence time prediction model to predict the aging time of the components. For components with unknown aging time, the image of the component can be input into the trained event occurrence time prediction model. The first few layers of the first convolutional layer of the event occurrence time prediction model can extract low-dimensional features and local features in the input 2D / 3D image, and can obtain image features of local image regions, that is, image features of aged components, such as aging area, color, etc. With the expansion of the receptive field, the deep convolutional layer will further extract abstract features and global features based on the image features output by the first few layers of the convolutional layer. The second convolutional layer of the event occurrence time prediction model can be the last few layers of the neural network model, which can obtain global features of local image regions. Global features can be used to represent image features of local image regions in multiple dimensions, such as shape, size, color, position, etc. The second convolutional layer of the event occurrence time prediction model can represent image features of multiple dimensions such as shape, size, color, position, etc. with global features, that is, global features can be used to represent image features of local image regions in multiple dimensions such as shape, size, color, position, etc. Image features of multiple dimensions such as shape, size, color, position, etc. can be used to represent the progress of the event, for example, the larger the aging area and the darker the color of the component, the longer the aging time of the component.

[0050] In the embodiment, the neural network model is trained by using the images of the aged components, and an event occurrence time prediction model is obtained. The local image area of the aged component is extracted by the event occurrence time prediction model, and the image features of the local image area of the aged component are analyzed. The image features of the aged component in multiple dimensions such as shape, size, and color are represented by the global features, and the aging time of the component is predicted by the global features. The problem of low efficiency caused by the need to rely on expert experience to predict the aging time of the component in the prior art can be solved, and the technical effect of improving the efficiency of predicting the aging time of the component can be achieved through the above embodiment.

[0051] As an optional implementation, the application can also predict the onset time of a patient or an animal. The onset time can be the onset duration or the time point of the onset. The onset duration refers to the time interval from the time point of the onset to the time of shooting the target image. Taking the prediction of the onset duration of a patient as an example, in an actual medical scene, an important reference information for a doctor to formulate a treatment plan is the specific onset duration of the patient. The onset duration not only affects the treatment difficulty and surgical risk of the patient, but also affects the prognosis of the patient. However, in some scenarios, the patient may lose consciousness or be unaccompanied when the disease occurs, so it is impossible to provide an accurate onset duration during diagnosis and treatment. The method of predicting the disease onset duration by using deep learning and artificial intelligence in the embodiment can predict the real onset duration of the patient at a high accuracy level, thereby assisting the doctor in making customized medical treatment decisions. The target object can be an organ of the patient, such as the head, chest, etc. The organ can be photographed to obtain a photographed image of the organ using the existing technology, such as a CT image or a B-ultrasound image, etc. Since the accurate position of the patient's onset cannot be known in a normal case, the overall organ of the onset site of the patient is usually photographed, the image features of the local image area of the onset site are extracted by the event occurrence time prediction model, the image features of the local image area in multiple dimensions are represented by the global features, and the onset duration or the time point of the onset of the patient can be predicted based on the global features. The onset duration refers to the time interval from the time of the onset of the patient to the time of shooting the image.

[0052] As shown in Figure 4 The patient onset duration prediction process according to the optional embodiment of the application is shown in the flowchart. The prediction of the onset duration of the patient can include the following steps:

[0053] In step S41, images of organs with different onset durations are obtained, and the corresponding onset durations are labeled. The photographed image can be a medical scan image, such as a CT image, as shown in Figure 5Fig. 1 shows a brain CT image according to an optional embodiment of the present application, and the areas represented by arrows S1, S2 and S3 are the diseased areas. In this embodiment, medical scan images of a plurality of patients and the onset times of the patients corresponding to the images can be collected as a training data set and a verification data set of the network. The training data set is used to train and fit the neural network, and the verification set is used to select the network model with the best generalization performance during the training process.

[0054] In step S42, data augmentation processing is performed on the data set using random flipping, rotation and other methods to enrich the diversity of the data. Since the image data of medical images is often difficult to obtain, the amount of available data is small, and the network cannot be effectively trained. In order to solve this problem, random flipping, rotation and other methods are used in this embodiment to perform data augmentation processing on the data set to enrich the diversity of the data and improve the network training effect.

[0055] In step S43, the deep learning network model is initialized. Before starting to train the network model, the network parameters are initialized using a random initialization method.

[0056] In step S44, the model is trained using the training data set. When the input medical image is a 2D image (such as an X-ray scan), a neural network model such as VggNet or ResNet can be used to process the input image. For example, the VggNet-19 network structure shown in Fig. 2 can be used to process the input image. Figure 6 Fig. 2 shows a VggNet-19 network structure according to an optional embodiment of the present application. Figure 7 Fig. 3 shows a ResNet-34 network structure according to an optional embodiment of the present application. When the input medical image is a 3D image (such as a CT image or a magnetic resonance imaging MRI), the 2D convolutional layer in the neural network can be replaced by a 3D convolutional layer, and a corresponding 3D VggNet or 3D ResNet network structure can be used to process the input image. In the neural network of this embodiment, the first few convolutional layers are used to extract low-dimensional features and local features from the input 2D / 3D image. As the receptive field expands, the deep convolutional layers will further extract abstract features and global features based on the output features of the first few convolutional layers. Finally, a fully connected layer is used to integrate the features obtained by the last convolutional layer to obtain a scalar output. In the training process, the input of the network is the medical scan image of the patient, and the scalar output of the network corresponds to the true onset time of the patient. In this embodiment, MAE can be used as the loss function of the network, and Adam algorithm can be used as the optimization method of the network. After multiple iterations, when the model has been basically fitted to the training data, the best network model based on the pre-prepared verification set is selected as the event occurrence time prediction model.

[0057] Step S45, using the trained event occurrence time prediction model to predict the onset duration of the patient. After training, the trained neural network model can be used to input the 2D / 3D medical scan image of the patient with unknown onset duration into the network, and the output scalar of the last fully connected layer of the network is used as the predicted value of the patient's onset duration. Specifically, after obtaining the input image, the network first extracts the features in the image through multiple convolutional layers. The first few convolutional layers extract local features in 2D / 3D images, and subsequent deep convolutional layers further integrate global features that have strong generalization for images based on local features. After extracting the features, the network finally integrates the feature vector obtained by the last convolutional layer using a fully connected layer to output a numerical scalar result. The numerical result is the predicted value of the patient's actual onset duration.

[0058] In this embodiment, the neural network model is trained using the patient's medical images, and the trained event occurrence time prediction model is used to predict the image with unknown onset duration, which saves the time of predicting the onset duration and improves the efficiency of the doctor's diagnosis and treatment of the patient.

[0059] Optionally, the target image is input into the event occurrence time prediction model, and the first convolutional layer in the event occurrence time prediction model is used to extract image features from the target image to obtain local image features matched with the local image region, including: identifying the local image region in the target image through the first convolutional layer, wherein the local image features include image features of the local image region in multiple dimensions; determining the image features of the local image region in the multiple dimensions through the first convolutional layer, wherein the target global feature is used to represent the image features of the local image region in multiple dimensions, the multiple dimensions are matched with the target event, and the image features in the multiple dimensions are used to represent the progress of the target object in the target event.

[0060] As an optional implementation, the acquired image usually contains other parts of the target object. For example, when predicting the aging time of the component on the device, the component on the device is photographed, and the image may contain other parts of the device. In this case, the local image region of the aging component in the target image is needed, and the local image region is the region of the aging component in the image. For another example, due to the inability to accurately predict the position of the patient's disease, a certain range around the position of the patient's disease needs to be photographed. For example, when the patient has a headache, the entire brain of the patient needs to be photographed, and when the patient has a cough, the entire lung of the patient needs to be photographed. Most of the photographed images do not have the disease. In order to better predict the onset time of the disease, the local image region of the disease part can be identified, and the local image region is the region where the disease part is located.

[0061] As an optional implementation, the influencing event occurrence time can be a plurality of dimensions of factors, and the plurality of dimensions can be size, color, position, shape, etc. Figure 8 and Figure 9 are schematic diagrams of disease part images of different onset times. The areas pointed to by arrows S1, S2, and S3 in the figure are disease areas. As can be seen from the figure, the longer the onset time, the larger the area of the disease part, the darker the color, the closer the disease position to the important organ position of the patient, and the more the shape of the disease part extends to other parts. Therefore, the image features of the disease part can be analyzed in combination with the size, position, shape, color, and other dimensions of the disease part, which can improve the accuracy of predicting the onset time of the patient.

[0062] As an optional implementation, the factors that usually affect the device aging time can be size, color, etc. For example, for metal components, the longer the aging time, the larger the range of oxidation, such as Figure 10 and Figure 11 are schematic diagrams of metal surface features of different aging times. As can be seen from the figure, the longer the aging time, the larger the area of the metal surface that is oxidized, and the darker the color. Therefore, the size dimension and the color dimension can be used as a basis for judging the aging time of the metal component. As an optional implementation, the factors that affect the patient's onset time can be the area, color, position, and shape of the disease part. In this embodiment, by analyzing the plurality of dimensions of the local image region in the photographed image, the accuracy of predicting the aging time of the component can be improved.

[0063] Optionally, the event occurrence time information of the target object occurring the target event is determined based on the target global feature, including: inputting the target global feature into a full connection layer of the event occurrence time prediction model to obtain a predicted occurrence duration output by the full connection layer, wherein the predicted occurrence duration represents a duration between a time point at which the target object starts to occur the target event and a time point at which the target image is captured, and the event occurrence time information includes the predicted occurrence duration.

[0064] As an optional implementation, the event occurrence time information can be a duration between a time point at which the target object starts to occur the target event and a time point at which the target image is captured. The full connection layer of the event occurrence time prediction model integrates the global feature obtained by the last convolution layer to obtain a scalar output, which is used to represent an output result of the event occurrence time prediction model. The predicted occurrence duration is obtained by the event occurrence time prediction model and is used to represent a duration between a time point at which the target object starts to occur the target event and a time point at which the target image is captured. For example, a duration between a time point at which the aging event occurs and a time point at which the image containing the aging component is captured. For another example, a duration between a time point at which the patient starts to be ill and a time point at which the image is captured. Assuming that the illness duration of the patient is predicted, the illness duration of the patient can be determined according to the area size, color, position and shape of the illness part of the patient. The target global feature can be used to represent the features of the illness part (local image area) in multiple dimensions such as shape, size and color. The target global feature is input into the full connection layer of the event occurrence time prediction model, and the illness duration of the patient can be predicted by the full connection layer, that is, a duration between a time point at which the patient starts to be ill and a time point at which the image is captured. Through this embodiment, the illness duration of the patient can be predicted, which can assist doctors in diagnosing and treating the patient and improve the diagnosis and treatment efficiency of the medical treatment.

[0065] Optionally, the event occurrence time information of the target object occurring the target event is determined based on the target global feature, including: inputting the target global feature into a full connection layer of the event occurrence time prediction model to obtain a predicted occurrence duration output by the full connection layer, wherein the predicted occurrence duration represents a duration between a time point at which the target object starts to occur the target event and a time point at which the target image is captured; and determining a predicted occurrence time point based on the predicted occurrence duration and a pre-acquired capturing time point, wherein the capturing time point is a time point at which the target image is captured, the predicted occurrence time point represents a time point at which the target object starts to occur the target event, and the event occurrence time information includes the predicted occurrence time.

[0066] As an optional implementation, the event occurrence time information can be a time point at which the target event starts to occur. The fully connected layer of the event occurrence time prediction model integrates the global features obtained by the last convolutional layer to obtain a scalar output, which is used to represent the output result of the event occurrence time prediction model. The predicted occurrence duration is obtained by predicting the event occurrence time prediction model, and the predicted occurrence duration is used to represent the duration between the time point at which the target object starts to occur the target event and the time point at which the target image is captured. For example, the duration between the time point at which the aging event occurs and the time point at which the image containing the aging component is captured. For another example, the duration between the time point at which the patient starts to be ill and the time point at which the image is captured. Based on the predicted occurrence duration, the time point at which the target object starts to occur the target event can be determined. Assuming that the onset time of the patient is predicted, the target global feature can be used to represent the features of the diseased area (local image area) in multiple dimensions such as shape, size, color, etc. The target global feature is input into the fully connected layer of the event occurrence time prediction model, and the onset duration of the patient can be predicted by the fully connected layer, that is, the duration between the time point at which the patient starts to be ill and the time point at which the image is captured. The time point at which the patient image is captured is known, for example, the image capture time point is recorded on the CT image or B-ultrasound image. Based on the predicted duration and the image capture time point, the onset time of the patient can be obtained, for example, the predicted onset duration of the patient is 3 days, the image capture time point is January 4, 2020, and the onset time of the patient is determined to be January 3, 2020. By this embodiment, the onset time of the patient can be predicted, which can assist doctors in diagnosing and treating patients and improve the efficiency of medical diagnosis and treatment.

[0067] As an optional implementation, the event occurrence time prediction model can identify the local features of the image and extract multiple dimensions of the local features, such as size, shape, position, color, etc. For example, the local feature of the image can be represented as a vector, and the vector can be input into the event occurrence time prediction model to predict the time point at which the target event starts to occur. Figure 12As shown in the figure, the image taken of the patient is input to the neural network model, the neural network model can extract the low-dimensional features and local features in the input image to obtain a local image, and the event occurrence time prediction model can extract multiple dimensional image features in the local image, such as T-1, T-2, … T-N in the figure, which can represent size, shape, position, or color, etc. The neural network model integrates the multiple dimensional features T-1, T-2, … T-N to obtain the global feature Ty of the local image, and the global feature Ty can represent the multiple dimensional local features T-1, T-2, … T-N, and the actual occurrence time of the event can be predicted according to the local feature Ty. In this embodiment, the image features can be extracted by the neural network model, and the occurrence time of the event can be predicted by the extracted image features, which can improve the accuracy. In this embodiment, the local image of the taken image can be extracted by the neural network model, and the multi-dimensional features of the local image can be analyzed, which can avoid the low efficiency problem caused by manual annotation of the region of interest in the prior art. The application can automatically extract the region of interest in the image by the neural network model, and improve the efficiency of the event occurrence time.

[0068] Optionally, the above-mentioned inputting the above-mentioned target global feature into the fully connected layer of the above-mentioned event occurrence time prediction model to obtain the predicted occurrence time length output by the fully connected layer includes: determining the prediction probability of each time length in a preset time length set through the fully connected layer; and determining the predicted occurrence time length in the time length set according to the prediction probability of each time length.

[0069] As an optional implementation, as shown in the figure, Figure 12 As shown in the figure, the fully connected layer in the neural network model can predict the onset time of the event according to the global feature, and the output result can be a probability value in different time ranges, for example Figure 12Ty1', Ty2',..., Tym' can represent different time lengths respectively, for example, Ty1' can represent one week, that is, the time distance between the invention time and the time of shooting the image is one week, Ty2' represents two weeks, and Tym' represents three weeks. The neural network model obtains the probability values of Ty1', Ty2',..., and Tym'. In this embodiment, the predicted occurrence time of the event is determined in the preset time length set. Since the preset time length set is used in training, different time length sets can be set for different training targets. Assuming that the onset time is predicted, the neural network model obtains a probability of 10% for the onset time being Ty1' (one week), a probability of 60% for the onset time being Ty2' (two weeks), and a probability of 30% for the onset time being Ty3' (two weeks). Then, it can be determined that the time corresponding to the maximum probability value is the onset time, that is, the onset time is two weeks. If the patient's imaging time is March 29, 2020, it can be inferred that the actual onset time of the patient is March 15. In this embodiment, the probability values of the patient in different onset time lengths can be obtained through the full connection layer, and the time length with the maximum probability value is selected as the onset time length, which can improve the accuracy of predicting the actual occurrence time length of the event.

[0070] Optionally, the method further includes: obtaining a sample image set, wherein the sample image set includes a sample image and known event occurrence time information corresponding to the sample image, the sample image is an image obtained by shooting a sample object, and the known event occurrence time information includes an actual time point at which the sample object occurs the target event or a time length between the actual time point at which the sample object occurs the target event and a time at which the sample image is shot; and training a sample neural network model using the sample image set until a target loss value of the sample neural network model meets a target convergence condition to obtain the event occurrence time prediction, wherein the target loss value is a loss value determined by event occurrence time information output by the sample neural network model and the known event occurrence time information.

[0071] As an optional implementation, an initial neural network model can be trained using a sample image set, and after repeated adjustment of model parameters, a trained event occurrence time prediction model can be obtained. The predicted time output by the event occurrence time prediction model and the known time of the image meet a predetermined convergence condition. The predetermined convergence condition can be that an output value of a convergence function of the predicted time output by the event occurrence time prediction model and the known time of the image is within a preset range. For example, the convergence function can be a function of the predicted time and the known time, and the output value of the convergence function can be a difference between the predicted time and the known time. Figure 13is a training schematic diagram of an event occurrence time prediction model according to an optional embodiment of the present application, wherein input sample images P1, P2, …, Pi, the sample images can be images of aged components or images taken of the onset position of a patient. The output of the model is the predicted duration Ty1', Ty2', …, Tyi', and the actual duration of the sample is known Ty1, Ty2, …, Tyi. Taking the prediction of the onset time of a patient as an example, the above-mentioned sample images P1, P2, …, Pi can be images taken of the onset position of a patient with known onset time, corresponding to the images P1, P2, …, Pi, the known onset time is Ty1, Ty2, …, Tyi. The initial neural network model is trained for the jth round using the sample images P1, P2, …, Pi, to obtain the jth round of estimated onset time output by the jth round of neural network model, calculate the loss value between the jth round of estimated onset time output by the jth round of neural network model and the known onset time, if the loss value meets the predetermined convergence condition, stop training to obtain the event occurrence time prediction model, if the loss value does not meet the predetermined convergence condition, perform the j+1th round of training, until the output result of the j+1th round of neural network model and the known onset time meet the predetermined convergence condition, wherein j is an integer. In this embodiment, the model is trained using effective training data, the model parameters are adjusted through repeated training of the model, and an accurate prediction model is obtained, and a more accurate event prediction occurrence time can be obtained through the trained prediction model. In addition, the obtained event occurrence time prediction model can predict the actual occurrence time of the event, and can improve the efficiency of predicting the occurrence time of the event.

[0072] Optionally, the above-mentioned obtaining the above-mentioned sample image set comprises: obtaining a first sample image, a first actual occurrence time point corresponding to the first sample image, a second sample image and a target duration, wherein the first sample image and the second sample image are images taken of a first sample object, the first actual occurrence time point represents an actual time point of occurrence of the target event of the sample object, and the target duration represents a duration between a time point of taking the first sample image and a time point of taking the second sample image; determining a second actual occurrence time point corresponding to the second sample image according to the first actual occurrence time point and the target duration, wherein the second actual occurrence time point represents an actual time point of occurrence of the target event of the sample object; wherein the sample image set comprises the first sample image and the first actual occurrence time point having a corresponding relationship, and the second sample image and the second actual occurrence time point having a corresponding relationship.

[0073] As an optional implementation, a large amount of training data is required to train the neural network model, and the image data of some categories is often difficult to obtain, and the amount of available data is generally small, such as medical image data. In this embodiment, the same sample object can be photographed at different times, and the event occurrence time at a certain photographing time can be used to calculate the event occurrence time at other photographing times, so as to obtain the actual event occurrence time corresponding to the images of the sample photographed at different times, thereby increasing the amount of training data. For example, for a patient with ischemic stroke, a first sample image P1 is obtained by photographing at a first time t1, and the actual onset time of the first sample image is determined by a doctor to be h days. The second sample image P2 of the patient with ischemic stroke is obtained by photographing at the second time t2, and the actual onset time corresponding to the second sample image is unknown to the doctor. In this embodiment, the time interval f = t2-t1 between the first time t1 and the second time t2 can be calculated, and the actual onset time of the second sample image P2 can be calculated as f+h. Through this embodiment, the amount of training data can be increased, and the accuracy of the model in predicting the actual event occurrence time can be improved.

[0074] Optionally, the above-mentioned obtaining the above-mentioned sample image set comprises: obtaining a third sample image and a third actual occurrence time point corresponding to the third sample image, wherein the third actual occurrence time point represents an actual time point at which a second sample object occurs the target event; performing random flipping and / or rotation processing on the third sample image to obtain a fourth sample image; determining that an actual occurrence time point corresponding to the fourth sample image is the third actual occurrence time point; and wherein the sample image set comprises the third sample image and the third actual occurrence time point having a corresponding relationship, and the fourth sample image and the third actual occurrence time point having a corresponding relationship.

[0075] As an optional implementation, the third sample image is an image of a known event occurrence time point, for example, a patient image of a known onset time point or a component image of a known aging time. In order to increase the amount of training data, the third sample image can be subjected to image random flipping and rotation processing, and the actual event occurrence time point of the third sample image can be used as the actual event occurrence time point of the fourth sample image after image processing. Using the third sample image and the corresponding event occurrence time point, and the fourth sample image and the corresponding actual event occurrence point as the training data of the neural network model can increase the amount of training data and improve the accuracy of the event occurrence time prediction model in predicting the event occurrence time.

[0076] The application provides a brand-new event actual occurrence time prediction method, which can be applied to any event actual occurrence time prediction scene, for example, predicting the aging time of an aging device, predicting the aging time of any type of device, or predicting the aging time of any component on the device. The application can also be applied to predicting the onset time of a patient, which can be applied to any disease, such as the onset time of ischemic stroke, the onset time of cancer, and the like. In the scenario where the patient's onset time is unknown, the application can provide effective reference information for the diagnosis and treatment process of medical personnel. In the application, the neural network model can learn the local features of the image automatically, and the region of interest can be extracted automatically without manual delineation of the region of interest. It is more convenient and has strong flexibility. According to different requirements, there are various methods in the neural network model for selection.

[0077] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0078] According to another aspect of the embodiments of the present application, an image processing apparatus for implementing the above-mentioned image processing method is also provided. As shown in the Figure 14 The apparatus comprises: an acquisition module 1402 configured to acquire a target image of a target object, wherein the target image comprises a local image region where a target event occurs; a first input module 1404 configured to input the target image into an event occurrence time prediction model, perform image feature extraction on the target image through a first convolutional layer in the event occurrence time prediction model, and obtain a local image feature matched with the local image region, wherein the local image feature is used to represent the progress of the target event of the target object; a second input module 1406 configured to input the local image feature into a second convolutional layer of the event occurrence time prediction model, and obtain a target global feature output by the second convolutional layer; and a determination module 1408 configured to determine event occurrence time information of the target event of the target object based on the target global feature.

[0079] Optionally, the apparatus is further configured to implement the inputting of the target image into the event occurrence time prediction model by performing image feature extraction on the target image through a first convolutional layer in the event occurrence time prediction model, to obtain local image features matching the local image region, wherein the local image features comprise image features of the local image region in multiple dimensions; and determining the image features of the local image region in the multiple dimensions through the first convolutional layer, wherein the target global feature is used to represent the image features of the local image region in the multiple dimensions, the multiple dimensions match the target event, and the image features in the multiple dimensions are used to represent a progress of the target object in the target event.

[0080] Optionally, the apparatus is further configured to implement the determining of the event occurrence time information of the target object in the target event based on the target global feature by inputting the target global feature into a fully connected layer of the event occurrence time prediction model, to obtain a predicted occurrence duration output by the fully connected layer, wherein the predicted occurrence duration represents a duration between a time point at which the target object starts the target event and a time point at which the target image is captured, and the event occurrence time information comprises the predicted occurrence duration.

[0081] Optionally, the apparatus is further configured to implement the determining of the event occurrence time information of the target object in the target event based on the target global feature by inputting the target global feature into a fully connected layer of the event occurrence time prediction model, to obtain a predicted occurrence duration output by the fully connected layer, wherein the predicted occurrence duration represents a duration between a time point at which the target object starts the target event and a time point at which the target image is captured; and determining a predicted occurrence time point based on the predicted occurrence duration and a pre-acquired capture time point, wherein the capture time point is the time point at which the target image is captured, the predicted occurrence time point represents the time point at which the target object starts the target event, and the event occurrence time information comprises the predicted occurrence time.

[0082] Optionally, the apparatus is further configured to implement the inputting of the target global feature into a fully connected layer of the event occurrence time prediction model, to obtain a predicted occurrence duration output by the fully connected layer, by determining, through the fully connected layer, a prediction probability of each duration in a preset duration set; and determining the predicted occurrence duration in the duration set according to the prediction probability of each duration.

[0083] Optionally, the apparatus is further configured to obtain a sample image set, wherein the sample image set comprises a sample image, and known event occurrence time information corresponding to the sample image, the sample image is an image obtained by photographing a sample object, and the known event occurrence time information comprises an actual time point at which the sample object occurs the target event, or a time length between the actual time point at which the sample object occurs the target event and a time at which the sample image is photographed; and train a sample neural network model using the sample image set until a target loss value of the sample neural network model meets a target convergence condition, to obtain the event occurrence time prediction, wherein the target loss value is a loss value determined by event occurrence time information output by the sample neural network model and the known event occurrence time information.

[0084] Optionally, the apparatus is further configured to obtain the sample image set in the following manner: obtaining a first sample image, a first actual occurrence time point corresponding to the first sample image, a second sample image, and a target time length, wherein the first sample image and the second sample image are images obtained by photographing a first sample object, the first actual occurrence time point represents an actual time point at which the sample object occurs the target event, and the target time length represents a time length between a time point at which the first sample image is photographed and a time point at which the second sample image is photographed; determining a second actual occurrence time point corresponding to the second sample image according to the first actual occurrence time point and the target time length, wherein the second actual occurrence time point represents an actual time point at which the sample object occurs the target event; and wherein the sample image set comprises the first sample image and the first actual occurrence time point having a corresponding relationship, and the second sample image and the second actual occurrence time point having a corresponding relationship.

[0085] Optionally, the apparatus is further configured to obtain the sample image set in the following manner: obtaining a third sample image and a third actual occurrence time point corresponding to the third sample image, wherein the third actual occurrence time point represents an actual time point at which a second sample object occurs the target event; performing random flipping and / or rotation processing on the third sample image to obtain a fourth sample image; determining that an actual occurrence time point corresponding to the fourth sample image is the third actual occurrence time point; and wherein the sample image set comprises the third sample image and the third actual occurrence time point having a corresponding relationship, and the fourth sample image and the third actual occurrence time point having a corresponding relationship.

[0086] According to still another aspect of the embodiments of the present application, an electronic device for implementing the image processing method is further provided, and the electronic device can be Figure 1The terminal device or server shown. This embodiment takes the electronic device as a server as an example. As shown in Figure 15 The electronic device includes a memory 1502 storing a computer program and a processor 1504 configured to execute the steps of any of the above method embodiments through the computer program.

[0087] Optionally, in this embodiment, the above-mentioned electronic device can be located in at least one of the plurality of network devices of the computer network.

[0088] Optionally, in this embodiment, the above-mentioned processor can be configured to execute the following steps through the computer program:

[0089] S1, obtaining a target image of a target object, wherein the target image includes a local image area where a target event occurs;

[0090] S2, inputting the target image into an event occurrence time prediction model, performing image feature extraction on the target image through a first convolutional layer in the event occurrence time prediction model, and obtaining a local image feature matched with the local image area, wherein the local image feature is used to represent the progress of the target object in the target event;

[0091] S3, inputting the local image feature into a second convolutional layer of the event occurrence time prediction model, and obtaining a target global feature output by the second convolutional layer;

[0092] S4, determining event occurrence time information of the target object in the target event based on the target global feature.

[0093] Optionally, those skilled in the art can understand that, Figure 15 The structure shown is only schematic, and the electronic device can also be a terminal device such as a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, etc. Figure 15 It does not limit the structure of the above-mentioned electronic device. For example, the electronic device can also include more or less components (such as network interfaces, etc.) than Figure 15 shown, or have a different configuration than Figure 15 shown.

[0094] The memory 1502 can be used to store software programs and modules, such as program instructions / modules corresponding to the image processing method and device in the embodiments of the present application. The processor 1504 executes various functions and data processing by running the software programs and modules stored in the memory 1502, that is, implements the image processing method described above. The memory 1502 can include a high-speed random access memory, and can also include a non-volatile memory such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 1502 can further include a memory remotely arranged with respect to the processor 1504, which can be connected to the terminal through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. Specifically, the memory 1502 can include but is not limited to information such as a target image obtained by photographing a target object. As an example, as shown in Figure 15 The memory 1502 can include but is not limited to the acquisition module 1402, the first input module 1404, the second input module 1406, and the determination module 1408 in the image processing device described above. In addition, other module units in the image processing device described above can also be included, but not limited to, which will not be described in detail in this example.

[0095] Optionally, the transmission device 1506 is used to receive or send data via a network. Specific examples of the above-mentioned network can include wired networks and wireless networks. In one example, the transmission device 1506 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers through a network cable to communicate with the Internet or a local area network. In one example, the transmission device 1506 is a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet in a wireless manner.

[0096] In addition, the electronic device further includes a display 1508 for displaying the target image obtained by photographing the target object, and a connection bus 1510 for connecting various module components in the electronic device.

[0097] In other embodiments, the terminal device or the server can be a node in a distributed system, where the distributed system can be a blockchain system, which can be a distributed system formed by the plurality of nodes communicating through a network. The nodes can form a peer-to-peer (P2P, Peer To Peer) network, and any form of computing device, such as a server, a terminal, and other electronic devices, can become a node in the blockchain system by joining the peer-to-peer network.

[0098] According to an aspect of the present application, a computer program product or computer program is provided, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the methods provided in the various optional implementation manners described above. Wherein, the computer program is configured to execute the steps in any of the method embodiments described above when executed.

[0099] Optionally, in the embodiment, the computer readable storage medium described above can be configured to store a computer program for executing the following steps:

[0100] S1, obtaining a target image of a target object, wherein the target image comprises a local image region in which a target event occurs;

[0101] S2, inputting the target image into an event occurrence time prediction model, performing image feature extraction on the target image through a first convolutional layer in the event occurrence time prediction model, and obtaining a local image feature matched with the local image region, wherein the local image feature is used to represent the progress of the target object in the target event;

[0102] S3, inputting the local image feature into a second convolutional layer of the event occurrence time prediction model, and obtaining a target global feature output by the second convolutional layer;

[0103] S4, determining event occurrence time information of the target object in the target event based on the target global feature.

[0104] Optionally, in the embodiment, those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer readable storage medium, and the storage medium can include a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0105] The serial numbers of the embodiments of the present application described above are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0106] The integrated units in the above embodiments, if implemented in the form of software function units and sold or used as independent products, can be stored in the above computer-readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions to make one or more computer devices (which can be personal computers, servers or network devices, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application.

[0107] In the above embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0108] In several embodiments provided in the present application, it should be understood that the disclosed client can be implemented by other ways. Among them, the above-described device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed units can be indirect coupling or communication connection through some interfaces, units or modules, and can be electrical or other forms.

[0109] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment according to actual needs.

[0110] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software function unit.

[0111] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. An image processing method, characterized by, The method comprises: acquiring a target image of a target object, wherein the target image comprises a local image region in which a target event occurs; inputting the target image into an event occurrence time prediction model, performing image feature extraction on the target image through a first convolutional layer in the event occurrence time prediction model, and obtaining local image features in multiple dimensions matched with the local image region, wherein the local image features are used to represent the progress of the target object in the target event; inputting the local image features into a second convolutional layer of the event occurrence time prediction model, and obtaining target global features of the local image region output by the second convolutional layer, wherein the target global features are used to indicate image features in each of the multiple dimensions of the local image region; integrating the image features in each of the multiple dimensions indicated by the target global features in a fully connected layer of the event occurrence time prediction model to obtain a predicted occurrence duration of the target event; in the event occurrence time prediction model, determining event occurrence time information of the target object in the target event based on the predicted occurrence duration of the target event, wherein the predicted occurrence duration represents a duration between a time point at which the target object starts to occur the target event and a time point at which the target image is captured, and the event occurrence time information comprises the predicted occurrence duration.

2. The method of claim 1, wherein, inputting the target image into an event occurrence time prediction model, performing image feature extraction on the target image through a first convolutional layer in the event occurrence time prediction model, and obtaining local image features in multiple dimensions matched with the local image region, comprising: identifying the local image region in the target image through the first convolutional layer, wherein the local image features comprise image features of the local image region in the multiple dimensions; determining the image features of the local image region in the multiple dimensions through the first convolutional layer, wherein the multiple dimensions are matched with the target event, and the image features in the multiple dimensions are used to represent the progress of the target object in the target event.

3. The method of claim 1, wherein, The determination of the event occurrence time information of the target object in the target event based on the predicted occurrence duration of the target event further comprises: determining a predicted occurrence time point based on the predicted occurrence duration and a pre-acquired capturing time point, wherein the predicted occurrence duration represents a duration between a time point at which the target object starts to occur the target event and a time point at which the target image is captured, the capturing time point is the time point at which the target image is captured, the predicted occurrence time point represents the time point at which the target object starts to occur the target event, and the event occurrence time information comprises the predicted occurrence time.

4. The method of claim 1, wherein, The integration of the image features in each of the multiple dimensions indicated by the target global features in a fully connected layer of the event occurrence time prediction model to obtain a predicted occurrence duration of the target event comprises: determining a prediction probability of each duration in a preset duration set through the fully connected layer; According to the predicted probability of each time length, a time length in which the prediction occurs is determined from the set of time lengths.

5. The method according to any one of claims 1 to 3, characterized in that, The method further includes: obtaining a set of sample images, wherein the set of sample images includes a sample image, and known event occurrence time information corresponding to the sample image, the sample image is an image obtained by photographing a sample object, and the known event occurrence time information includes an actual time point at which the sample object occurs the target event, or includes a time length between the actual time point at which the sample object occurs the target event and a time at which the sample image is photographed; training a sample neural network model using the set of sample images until a target loss value of the sample neural network model meets a target convergence condition, to obtain event occurrence time prediction, wherein the target loss value is a loss value determined by event occurrence time information output by the sample neural network model and the known event occurrence time information.

6. The method of claim 5, wherein, The obtaining the set of sample images includes: obtaining a first sample image, a first actual occurrence time point corresponding to the first sample image, a second sample image, and a target time length, wherein the first sample image and the second sample image are images obtained by photographing a first sample object, the first actual occurrence time point represents an actual time point at which the sample object occurs the target event, and the target time length represents a time length between a time point at which the first sample image is photographed and a time point at which the second sample image is photographed; determining a second actual occurrence time point corresponding to the second sample image according to the first actual occurrence time point and the target time length, wherein the second actual occurrence time point represents an actual time point at which the sample object occurs the target event; wherein the set of sample images includes the first sample image and the first actual occurrence time point having a corresponding relationship, and the second sample image and the second actual occurrence time point having a corresponding relationship.

7. The method of claim 5, wherein, The obtaining the set of sample images includes: obtaining a third sample image and a third actual occurrence time point corresponding to the third sample image, wherein the third actual occurrence time point represents an actual time point at which a second sample object occurs the target event; performing at least one of the following operations on the third sample image to obtain a fourth sample image: performing random flipping processing on the third sample image to obtain the fourth sample image; performing rotation processing on the third sample image to obtain the fourth sample image; determining that an actual occurrence time point corresponding to the fourth sample image is the third actual occurrence time point; wherein the set of sample images includes the third sample image and the third actual occurrence time point having a corresponding relationship, and the fourth sample image and the third actual occurrence time point having a corresponding relationship.

8. An image processing apparatus characterized by comprising: includes: an obtaining module configured to obtain a target image of a target object, wherein the target image includes a local image region in which a target event occurs; The first input module is configured to input the target image into the event occurrence time prediction model, perform image feature extraction on the target image through a first convolutional layer in the event occurrence time prediction model, and obtain local image features in multiple dimensions matched with the local image region, wherein the local image features are used to represent the progress of the target object in the target event. The second input module is configured to input the local image features into a second convolutional layer of the event occurrence time prediction model, and obtain target global features of the local image region output by the second convolutional layer, wherein the target global features are used to indicate image features in each of the dimensions. The determination module is configured to determine event occurrence time information of the target object in the target event based on the predicted occurrence duration in the event occurrence time prediction model, wherein the predicted occurrence duration represents a duration between a time point at which the target object starts to occur the target event and a time point at which the target image is captured, and the event occurrence time information includes the predicted occurrence duration.

9. The apparatus of claim 8, wherein, The device is further configured to identify the local image region in the target image through the first convolutional layer, wherein the local image features include image features of the local image region in the multiple dimensions.

10. The apparatus of claim 8, wherein, The device is further configured to determine a predicted occurrence time point based on the predicted occurrence duration and a pre-acquired capturing time point, wherein the predicted occurrence duration represents a duration between a time point at which the target object starts to occur the target event and a time point at which the target image is captured, the capturing time point is the time point at which the target image is captured, the predicted occurrence time point represents the time point at which the target object starts to occur the target event, and the event occurrence time information includes the predicted occurrence time.

11. The apparatus of claim 8, wherein, The device is further configured to determine a predicted probability of each duration in a preset duration set through the full connection layer, and determine the predicted occurrence duration in the duration set according to the predicted probability of each duration.

12. The apparatus of any one of claims 8 to 10, wherein, The apparatus is further configured to obtain a sample image set, wherein the sample image set comprises a sample image, and known event occurrence time information corresponding to the sample image, the sample image is an image obtained by photographing a sample object, and the known event occurrence time information comprises an actual time point at which the sample object occurs the target event, or a time length between the actual time point at which the sample object occurs the target event and a time at which the sample image is photographed; and train a sample neural network model using the sample image set until a target loss value of the sample neural network model meets a target convergence condition, to obtain the event occurrence time prediction, wherein the target loss value is a loss value determined by event occurrence time information output by the sample neural network model and the known event occurrence time information.

13. The apparatus of claim 12, wherein, The apparatus is further configured to obtain a first sample image, a first actual occurrence time point corresponding to the first sample image, a second sample image, and a target time length, wherein the first sample image and the second sample image are images obtained by photographing a first sample object, the first actual occurrence time point represents an actual time point at which the sample object occurs the target event, and the target time length represents a time length between a time point at which the first sample image is photographed and a time point at which the second sample image is photographed; and determine a second actual occurrence time point corresponding to the second sample image according to the first actual occurrence time point and the target time length, wherein the second actual occurrence time point represents an actual time point at which the sample object occurs the target event; and wherein the sample image set comprises the first sample image and the first actual occurrence time point having a corresponding relationship, and the second sample image and the second actual occurrence time point having a corresponding relationship.

14. The apparatus of claim 12, wherein, The apparatus is further configured to obtain a third sample image and a third actual occurrence time point corresponding to the third sample image, wherein the third actual occurrence time point represents an actual time point at which a second sample object occurs the target event; and perform at least one of the following operations on the third sample image to obtain a fourth sample image: performing random flipping processing on the third sample image to obtain the fourth sample image; performing rotation processing on the third sample image to obtain the fourth sample image; and determining an actual occurrence time point corresponding to the fourth sample image as the third actual occurrence time point; and wherein the sample image set comprises the third sample image and the third actual occurrence time point having a corresponding relationship, and the fourth sample image and the third actual occurrence time point having a corresponding relationship.

15. A computer readable storage medium, characterized in that, The computer-readable storage medium comprises a stored program, wherein the program performs the method described in any one of claims 1 to 7 when executed. 16.An electronic device comprising a memory and a processor, the electronic device characterized by, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 7 by using the computer program.

Images